YEAR 1 
WERSSSy NO a LUD] a RS 


PRESCHOOL EXPANSION GRANT (PEG) 
Evaluation Report 


ACKNOWLEDGMENTS 


The PEG evaluation team greatly appreciates the careful guidance and review from the Massachusetts 
Department of Early Education and Care, in particular Senior Research Specialist Dr. Jocelyn Bowne, 
who was deeply involved with overseeing the research and reviewing the report. We also sincerely thank 
the participating district staff, ELP leadership, program directors, teachers, other program staff, and 
parents who were essential partners in this research. 


The listed authors of the report represent only a small part of the team involved in this research. We wish 
to acknowledge the contributions of the project quality advisor, Michael Lopez, as well as the capable 
data collection team led by Jackie Mendez, Maureen Sarna, and Rachel Luck with Djaniele Taylor, Renee 
Lamoreau, Katie Murphy, Lisa Setrakian, Michelle Blocklin, Marble Karuu, Faith Biegon, Samantha 
Burke, Alanah Hall, and members of the data analysis team including Alex Silverman, Tyler Morrill, 
Rian Watt, and Marissa Personette. We also want to acknowledge the team of dedicated staff hired to 
conduct classroom observations and child assessments. In addition, we thank Sarah Costelloe for serving 
as an additional senior reviewer, and Jan Nicholson and Erin Miles for their assistance in formatting this 
report. 


Abt Associates Year 1 Massachusetts Preschool Expansion Grant (PEG) Evaluation Report J pg. i 


CONTENTS 


EXCCULIVE SS UMIMALY, ssis5escescedscsecvasseeesentaceosssovesssedsccseveovetessusessanveoeeecdensassduevescedscebussansnevecesseoussonsassasessecesedes 1 
Major Findings 3 i23csstsescesntsjctaseessieel Sisint entation ei dat masini tea Leese 1 
Detailed Findings on Classroom Quality .........cecsescssecsseceseceseceseeeseeeeaeeeaeeeaaecaecsaessaeesaeeseeeeseeeenteees 2 
Detailed Findings on Children’s Kindergarten Readiness ......0....ccesssceecceseceeeeeeeeceeeeseeeeeeaeeaneeneeaes 3 
Detailed Findings on Implementation of PEG Quality Elements 20.0.0... ee eeeeseceeeceneeeeeeeeeeeeneeees 4 
Detailed Findings on Teacher and Parent Experiences ...........eeeceeeeeseeeseeeseecseeceseceseeeeeeeeeeneeeeneeees 6 
INExt Steps for: PEG bs stein siscssaacoceses besutes aasosiusad sossusaveatesstsdaxeseassteroutsendensnanstsatebesdsreas seuenadecstanaunedisle 7 

1. PNET OGTR ON issn sasscesnoovcssnssedndsiueonusonissuetsansnyauanateiud tiie onsbcuaneveuaset yutinaseded ehusepns dpapeveadsantyueinesevebaiaes 8 

2. Overview of the Massachusetts PEG Program ............scsscsssscscsssccsscsccsccssscesscssseccssscescsessseses 10 
2.1 Structure of the PEG Program... ee eseessecsecsseceseceseceseceseeeseeeseeeeaeeeaeeeaeecaaecaessaesnaeenaeens 10 
2.2 PEG Program Model and Rationale ......... cece eeseceseceseceeeceeeeeseeeeeeeceeeeaeeeaeeeaeecaecsaesnaeenaeens 11 
23° ‘Children Enrolled in PEG wsniaiisinnakisirnincticnuiiesi ah neiitianhaiabianaietes 14 

3: PEG: Evaliation: Over vit w ssisissassxcsseiccessavesscesseisdinssdcesnes spavensasvesscessndecassvbesacasvovedesuvenssegueeasesvsasen 16 
Sul. Research Quests cs ciscivs cocviiies <cseses catedlsca scales cece ae’ ia Base Cees Ae Ge eevee ea ia Baha Vea as Moeaiben es 16 
3.2 Data Sources, Sample Description, and Measures ...........:eeeeesesssecsseceseceseceseceeeeseeeseeeeeeeees 16 

3.2.1 Surveys, Interviews and Focus Groups ..........cesceseceseceseceseeeeeeeseeeeeeeeneeeneeeaeeeaeeenaes 17 
3.2.2. Classroom Observations ......scc...cehicsctcdsseaicedennscicacesascoeveacicdenasessseunssduesnassodeeaensoeee 18 
32.3 “Child ASséSsment. wis. ceisussaee. seit cs iesseteveseasedaanstdeceaseevaseiibe ste esaseotiasinasvaserascoimeeteees 18 
3:2.4- Other Available: Data vi iscccs.ceiiec. cc seas costvasciesseessctecssasts evtascaededseldentisoptetenaescevebincneee 20 
3.3. Approach to Data Analysis ....... ce ceeceseceseceseceseeeseeeseeesaeeeaeecaeecsaecsaecsaecsaecsaeeeseeseeeeseeeseeees 20 

4. Collaboration and Leadership ................ccsccccssscscssscsscssccesscessscscsssccssccssccsssccsscessessscccssccssssessseses 21 
Al Sumimary Of PIM gS iiss: cscsec eee cevseeea eceiads catieasecesbenssceetieaeredeieds ces bee Sapeteteecss ed caees 21 
4.3, Collaboration Models... ces cescesecssecsseeeseeesneesseeeseessaecaaecsaecaecsaecesessseeeseeeeeeseneeenaeenaeeaaes 22 

43:1 ‘BOStOD iiss iisciii dies dedi ihted tld cbensseiendisiinds eis den lehndiidieen beeen 22 
ASQ" HOLYOKE srt sc ctecents soecieateaadedeasetececlane st sv irene chodhieed duets dblasacdevsieaseudiuabdasehetelsies ive 22 
ABB LAWL CC cs casein cctesd inaceeeasianscensbbscuevis soteckens souks seuscdvasnsuteans og laebabecabenabenseeeddbaneeensseeee 22 
ADA, SMO WOM sa eseescct cts c a teese cn cases eden seetedasicaseite qa ainsi: 24euudsarisinatyauintessaceturssaeshepieeete-stcdees 23 
435 Sprame Gel. ciscecsiivisjscedeea si niceesetedesbed sSbelvesdusnndb bauiievas betes sd sgertedisenraaci boesasdedeeietsiee 23 
4.4 Collaboration Structure... ee eee eeeceseceseessseesseeeseeesaecaaecaeceaessseseaeeseeeseeeeeeseaeeeaeeaaeenaes 23 
4.5 Collaboration Experiences: Successes and Challenges... cescescescsseeeseeeeeeeseeeneeeeeenaes 24 
4.5.1 Successes of PEG Community Governance Structure 0.0... eeeeeeeseeseeeeeeeeeeeeee 24 
4.5.2. Sharing Best: Practices -siisc..sivevs.dccese cs Sievesieenas te vieesaedsueesslselieenlehindiibiveesberesesieente 24 
4.5.3 Alignment of Regulations and Policies... eee eeceeceseeeeeeeeeseeeseeeaecnaeenaeeeaeee 25 
BSA. WBA CROMG 355d ccutsecaiccedaaceuncassate edn tc ceesap ok lentes ecanndas sadeded Pav annd as Secane an danedasoaecagiezebaaan ste 25 
4.5.5 Communication among PEG Leaders 0... eee ceeceeeeeeeeeeeeeeseeeseeeseeeaesnaeenaeenaeees 25 
4.5.6 Communication within ELPS oo... ee eesecssecssecsseceeeeeseeseneeeaeeeseeeaaeeaesaeenaeenaeees 26 
4.5.7 Support for Center Directors 0... cece ceseesseceseceseceeeceseeeseeseneeeneeeseessaeesaessaessaeeeaeee 26 
4.6 Wilder Collaboration Factors Inventory Findings 00.0... cies ceseceeeeeeeeeseeeeeeeeneeeneeeaaeenaes 26 
4.6.1 Findings from the Wilder Collaboration Factors Inventory... ee eeeeeseeeseceeeees 27 


Abt Associates Year 1 Massachusetts Preschool Expansion Grant (PEG) Evaluation Report J pg. ii 


CONTENTS 


5. TOACHER SUP POLt is ccce seas sseciscecedecdsossctbesesesscensnsscessenssosdseus vocestes soeesdessasedenssessseetsobessucsosssesosessdevouseds 31 
Sl, Summary Of PMO Oss. cisccsiesctsstecesent cease star deaceslen sales vecusoateleees vectessstecsawsestetes ate seashidesieonns 31 
5.2. Charactenstics:of PEG Teachers? i ssciaitisnteaistaioiign elite ate ani karate 31 
DA- “COMPCHSAU OM: cscs: scsscastcescrnsteiesesaessrantsasenzes voudndaea suivegeressercdrescunessa uoctiasln dust ssnscousbaassuresisssee 33 
5.5 Professional DevelopMent .0........ eee eeeceseceseceseeeseeeeeeeeneeeneeeaeecacecaaecsaecsaecaecsaeenseceeeeeeeeeeneeees 33 

5.5.1 Hours of Professional Development .......... cece eeseceseceseceeeeeseeeseeeeeeeeeeeeaseeaeesaeeenaes 34 
5:9:2; “Vrawmine TOpics s.1sstasaiscisssssetains eateveseatesettathrs a ataedec Mei eaad ai sagen seid Ab ae sers 35 
5.5.3. Perceived Effectiveness of Professional Development ............ ee eeeeeeeeseereeeneeeneee 36 
5.5.4 Challenges in Implementing Professional Development ............eeeeeeeeeeneereeeeeee 37 
DG COACHING sesaciestes rarecacessees stn Meee seesede teases each em uanae deca at deus ede casateeeavleceates ihe daaceinesdieests 37 
9.0240 JPEG Coaches eis.csccncricts ne eisdeseeateititiine Gir ninbinnl Gabi aint 37 
9.6:2.° “COaching ACHViteSii. satoseensssscesssdscceesnasgutanta loversassuteanesenseessasesaassudesdaposcessaeeseressiaeeces 38 
5.6.3 ‘Coachitig: DOSa ge sctives setestitinsehaislstiess teats ete iesideeieens Solis aehaeeweeeie del 39 
5.6.4 Supervision and Documentation of Coaching......... cece ceeeeeceeeeeeeeeeeeeeeeeeeaeeenaes 40 
5.6.5 Effectiveness of Coaching ies. 2vsi:esessccdascasciisserstieatescnccieassieesanscell ssiensiessetenieeces 40 
5.6.6 Challenges Implementing Coaching .0...... cece eeseceseceseceeeeeeeeeseeeeeeeeneeeaeeeaeeeaeeenaes 41 
5.6.7 Coach-Recommended Improvement ............:cessessesseceseceseceeeeeseeeseeeeneeeaeeeaeeenaeenaes 41 
5.7 Paid Instructional Planning Time... eee eeeeseeeeeeeeeeeseeeaeecaaecsaecsaecsecsseesseeseeeeseeeeeeeees 41 
5.7.1. Teacher Perceptions of Planning Time 00.0.0... eee ceeceseceseceeeceseeeeeeeeneeeneeeseeeaeeenaes 42 
5.8 Early Teacher Outcomes 00.0... ce ceecesecsseceseceseceseeeseeeeneeeseeeaeeeaeecaaecsaecsaecesecsaeesseeeeeesseeeeeeeees 42 
5.8.1 Job Satisfaction and Self-Efficacy ..... eee eeessecesecsseceseceeceeeeeseeeseeeeseeeaeeeneeeaeeeaaes 42 
5.8.2 Teacher Perceptions of Support from PEG Administrators ......... ce eeeeeeeeseereeneee 44 
5.8.3. Additional Teacher Perceptions 00.0.0... ceeeeesecssecesecsseceseceseceeeeseeeeeeeeneeeneeeeeeaaeenaes 45 

6. Curriculum and. Learning oo. ccscsseccsssessoscssssasnssssooesssssconnssscssvosssoessecdsvossecscsossectseasooss soasseossvonas costes 47 
6.1 Summary Of Fins i..c.ccssccssccsseeacsevssovscecssasecaesensodasssnasseadenvs ce desscnasesauscvusenveceeasecadeesesetcuses 47 
G2 Curr CU foes ss cesecssdcctea sh casceiach casstacceiQeenscc vende candela shcavcidocbeceueeieevaussanecekees daweisopvatevasgeteuss onete 47 
6.3 Screening and ASSCSSMENt ....... eect ceseceseceseceseeeeeeeseeeeaeeeseecaeecaaecsaecsaecaecsaeeeseeseeeeseeeeeeees 49 
6.4 Classroom Activities and Groupings ....... eee eee eeeeeeceeeeeeseeescecaaecsaecsaeceaecsaeesseceeeeseeeeneeees 51 

6.4.1 Classroom ACtiVItICS 0... ee ee eee eeseeeseeesceesceeacecaaecsaecsaecaecsseeeseeeeeessneeeaeeeaeeeaeeeaaeenaee 51 
64.2: Child: Groupies. .stecscissscassseeit stein celteasiesnssnesil iva atediouivseeaila@enssote 53 

7 CTASSTOOM QUATIEY. os sccnscsisccsscoscsscsvsssccsses sososoosseessnes ce sosvosssosssecssensevsscussessssconnestnessesséssonaesseonoresesses 54 
FA Summary Of Findings wsscscscscisedicinasceasennesouessesencseuseduscsvsnseadevsesoussuvdousiduentaspsnsnenaseracceudsenisacds 54 
7.2 Measures of Classroom Quality 0... eeescesssecssecssecesecsecssecsseeeseesseeeeaeeeaeeeaeesaaecaaecsaeenaeenaeens 54 
7.3. Quality of Classroom Instruction in PEG Compared to Similar Programs............. cesses 57 

8. Family Engagement Activities and Comprehensive Services ............scsscssscssssssssssscssccsscseseseses 59 
Sl Sumimary Of Finding sic... cc ices ccscetens esseneticades is ceseebesaseviasssisvensesssuthas soteebontuseeusnaveeevadnetesnsesnte 59 

Family Engagement .00..... cece eeseessecssecesecesecsseeeseesseeeseessaeeeseeeneeesaecaaecnaecsaecaeesaeeseeseeeeeeeses 59 
Commperehenstve- Services's iicassiesessiiieesustedesecssdevdosssaseuiossedgeres todesees sbeveudssuonwsss+tonevas betes slsnee 60 
Kindergarten Transition ..........csceesecesecsseceseceseceseeeseessneseneeeseeeaeeesaecaaecsaecsaecsaeesseeseesseeeeeeeees 60 
Parent EXperiences si oésvessiceedevsci casei scvceseuatcavesdace-dubsosddecisosteteedesctvebis sa vebeesdsueissnveuevasseueea onde 60 
8.2 Family Engagement Activities 0.0... ce cesecesecsseceseceseceseeeseeeeaeecseecsaecsaecsaeceaeesseeseeeseeeseeeees 60 
8.2.1 Challenges in Engaging Families... eee eecceeceeereeeeeeseeeseesseecaecnaeceaeenaeeeaeees 62 
8.3 Comprehensive Services for PEG Families ......... cee eeeeseeseeeneeeeecseecsaecesecnseesseeeeeesseeseeeeees 62 


Abt Associates Year 1 Massachusetts Preschool Expansion Grant (PEG) Evaluation Report J pg. iii 


CONTENTS 


6.321 jiAssessme Pamily Ne€dS....2. scsissescbcsavesideenvessdesand saaziansdvessacdeacassufeanvaertedoeeaseemneesie 62 

8.3.2 Linking Families to Needed Services .0..... cee eeceeecesceeeeeeeeeeeseeeeeeeecneeenaecsaeenaeenaeens 62 

8.4 Services for Children with Special Needs... ee eeeeseeseeseeescecseecsaeceaeceaeceaeeeseceeeeeseeeseeeees 64 

8.5 Kindergarten Transition Supports ..0......... cee ceeceseeeeeeseeeeeeeeseecacecaaecaaeceaecaecsaeeeseeeeeeeeeeeeeeees 65 

8:6: Parent PXperemCes a ssc ssicciicvssveietenstditievssecutssduec eset ededidoicebeoevsdivanes opaaee aehedie Helienasdindenseeees 65 

8.6.1 Parent Connectedness to their Child’s PEG Program ..........cceeeeceeseeeeeeeeeseeneeeneeaes 65 

8.6.2. Parent Sel-EmMmCacy csiescoch woservet resetestnadecnniea ceeded eitealesiaa Sailing evaed ieitsi dee davneedece 66 

8.6.3 Home Support for Children’s Learning ..0....... ce eeeeesceeseeeeceeeeseeeeceaeeaeeeeeeseeaeeeneeaes 67 

9. CHG A SSOSSINI CNIS sis sssssassocesonovectasscnstenavenseseeposnnosastyaiendecnvdenocebausisesueussnusuntndesyetoiaueaesenssesueonensvousey 69 
OL. “SUMMALY ot edotses Cateies eiettat i idensesiaie lest tenet denen: ites astute eee ie ieee 69 

O:2. Barly Academic SKM S 5... sscdecssusesar venessersscidtorassedetendeosites ensetewsl isiteserdenacosutesoruscuenonalientesrwasese 69 

9.2.1 Academic Achievement and Home Language un... ese esseessecsseceseceeeceeeeeeeeeeeeeees 72 

9.3. Teacher Reports of Children’s Social-Emotional SKiIIs 00.0.0. .eeeeeeeceeseceeeeeeeeeceeeeeteereaeees 74 

10. DISCUSSION OF PINGING | .2.52:03ccc5socececsoccssessevesscvacessesssoiieecevtes sees setsssesedteseseseteseesodecseaesveasdemedecsescseses 76 
10:1. PEG Classroom: Quality sacccscvscocsdevetesiensiscevaves denn soni deeletecten ss hevedesddenedsianeedvasb eden nero beets 76 

10.2 Children’s Skills at End of Preschool .0.......ccccecescssessesseceeeeseeeceaecaeeeeeeaecaeeeeeeeecaeeneeeeeaeeas 76 

10.3. PEG Implementation Progress in the First Year... eseessesssecssecseceseceeceeeeseeeseeeseeeseeeees 78 

10.4 Teacher and Parent Experiences 00.0.0... ceeceecesceseeeseeeeeeeeaeeeseeeaeecsaecsaeceaecssecsaeesseeseeeeseeeeneeees 83 

10.5: (Next Steps:for PEG Evaluation iiss. csies.teeisisiseins discal al sbieeeistolesselsborndssviensssedanevaepeneesloniee 83 

PRETOP ENCES sc cocssescvaspossicispenuccbcwsmuccutyvasussapensecasnobasebsnausceutpusdnceopunseseenvonceshowsnsucponuasbopsmacseusvanessanunsacatppbneebiyyes 84 
Appendix A: Summary of Classroom Quality SCOreS ............csccsscssscssscssscesssescssscscsccssecssccssscssscssseessecees 89 
Appendix B: Summary of Teaching Strategies GOLD Objective and Skill Ratings...............ssssseees 92 


Abt Associates Year 1 Massachusetts Preschool Expansion Grant (PEG) Evaluation Report J pg. iv 


EXECUTIVE SUMMARY 


Executive Summary 


As part of the federal Preschool Development Grant Program in late 2014, the Massachusetts Department 
of Early Education and Care (EEC) was awarded a Preschool Expansion Grant (referred to as the 
Massachusetts PEG program) to expand high-quality early childhood education to four-year-old children 
from low-income families. The Massachusetts PEG program has provided the Commonwealth with a 
unique opportunity to increase access to high quality preschool through a mixed service delivery system. 
The PEG model is defined by the provision of key elements perceived to be important drivers of quality. 


Massachusetts is concentrating its PEG resources in five underserved high need communities: Boston, 
Holyoke, Lawrence, Lowell, and Springfield. These communities provide full-day, full-year preschool for 
four-year-olds through public-private partnerships between the lead education agency (LEA) and local 
licensed early learning providers (ELPs). In each community, the participating ELPs, including a Head 
Start agency and one or more community-based agencies, provide the services with support from the 
LEA. 


For families to be eligible for PEG, their income must be below 200 percent of the federal poverty level, 
and four out of the five communities also targeted four-year-olds that had not previously been enrolled in 
licensed child care settings. Beginning in September 2015, participating LEAs and ELPs in the five 
communities began implementing PEG in 48 four-year-old classrooms across 24 early education centers. 


As part of the PEG program, EEC has invested in a rigorous external evaluation. The PEG evaluation is 
being conducted by Abt Associates Inc. in partnership with University of Massachusetts Boston. The 
multi-year evaluation has four components: an implementation study of the provision of quality elements 
in PEG classrooms, a longitudinal study of outcomes for PEG educators, children and families, an impact 
study of effects of PEG on children and families, and a cost study. The first year of the PEG evaluation 
(2015-16) focused on the implementation of PEG. The impact study and longitudinal study will begin in 
the second year of PEG (2016-17) and the cost study will analyze costs over the first three years of PEG. 


Major Findings 
The following are this report’s major findings: 


e In the first year of implementation, the PEG classrooms, on average, demonstrated a moderate to high 
level of overall quality, with higher scores on quality of the classroom environment and lower scores 
on instructional quality. Quality varied substantially across classrooms. 

e At the end of the preschool year, PEG children, on average, demonstrated levels of early math skills, 
early literacy skills, and vocabulary comprehension close to the level shown in national samples of 
children entering kindergarten. However, the proportion of children meeting age expectations on 
these outcomes varied substantially across classrooms, and about one-third of PEG children overall 
were below age expectations on English language vocabulary. 

e All eleven key PEG quality elements were implemented to some degree in the first year of the 
program. Some quality elements were fully implemented in all communities and centers, while other 
PEG quality elements were partially implemented both within and across communities and programs. 

e PEG teachers reported a high level of job satisfaction and confidence at being able to teach and 
support the children in their classrooms. 
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e PEG parents reported a strong sense of connection to the program and satisfaction with the program’s 
support for their children’s development and learning. 


Detailed Findings on Classroom Quality 


The PEG classrooms, on average, represented a moderate to high level of overall instructional 
quality. 

Providing children with high-quality classroom instruction is a key goal of the PEG program. As of 
winter 2016, PEG classrooms, on average, received quality scores in the moderate to high range on the 
CLASS and the ELLCO, two commonly-used measures of classroom quality (see Exhibit S.1). As is 
typically observed in other preschool classrooms, on the CLASS PEG classrooms scored higher on 
Emotional Support and Classroom Organization (average ratings of 5.7 and 5.2, respectively) compared 
to Instructional Support (mean rating of 3.2). Quality varied widely across programs and classrooms. 


Exhibit S.1: Percent of PEG Classrooms with Low, Moderate and High Quality Ratings, 2015-16 


ELLCO: Classroom 


Environment Avg = 3.9 out of 


ELLCO: Language & Literacy Avg = 3.4 out of 5 


CLASS: Classroom 


Organization Avg = 5.2 out of 7 


Avg = 5.7 out of 7 


CLASS: Emotional Support 


CLASS: Instructional Support Avg = 3.2 out of 7 


0% 20% 40% 60% 80% 100% 
Low Quality Moderate Quality m= High Quality 


SAMPLE: N=48. 
SOURCE: PEG Classroom Observations (Winter 2016) using the CLASS PreK and the ELLCO. 
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Detailed Findings on Children’s Kindergarten Readiness 


At the end of the preschool year, PEG children demonstrated levels of early math skills, early 
literacy skills, and vocabulary comprehension close to what would be expected for children 
entering kindergarten. 

On average, at the end of preschool, PEG children scored near age expectation in their early literacy and 
early math skills; only about 12 percent of children were substantially below age expectation (see 
Exhibit S.2). Similarly, on average, PEG students scored near age expectation on English language 
vocabulary; however, although the average student’s vocabulary score was not far below what might be 
expected, about one-third of PEG students were substantially below age expectation at the end of 
preschool. Vocabulary scores were generally lower for students from homes where English was not the 
primary language spoken. The proportion of children meeting age expectations on all of the measures 
varied across classrooms. It is important to note that, because of the timing of the evaluation’s start, in 
2015-16 the study team was not able to measure where PEG children started out on these skills and how 
much they improved over the year. 


Exhibit S.2: Overall Distribution of PEG Children on Standardized Assessments, 2015-16 


N 


All Measures (National distribution) 2% 14% Avg. score =100 


Early Math (PEG distribution) 2 Avg. score =98. 1 


Early Literacy (PEG distribution) 1% 10% Avg. score =99.4 


Vocabulary (PEG distribution) Avg. score =90.6 


0% 20% 40% 60% 80% 100% 
VeryLow “Low wmExpected Range mHigh 


SAMPLE: N=324-326. 

SOURCE: PEG Child Assessment Data (Summer 2016) using the Woodcock-Johnson III Tests of Cognitive Abilities: Applied Problems 
Subtest for early math skills, the Woodcock-Johnson Ill Tests of Cognitive Abilities: Letter-Word Identification Subtest for early literacy skills 
and the Expressive One-Word Picture Vocabulary Test for vocabulary skills. 


Toward the end of their preschool year, PEG children were rated by their teachers as having age- 
appropriate social skills in managing behavior and emotions and in developing positive 
relationships. 

The types of skills involved included the ability to follow limits and expectations, manage feelings, and 
take care of their own needs. PEG children were rated somewhat lower on their skills in group situations, 
such as balancing their own needs with needs of other children or solving social problems. Further, 
though the average scores were in the moderate range, a substantial number of children were rated below 
age expectations for many of the individual skills, primarily in forming relationships with adults and 
solving social problems. 
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Detailed Findings on Implementation of PEG Quality Elements 


PEG achieved its goal of providing access to preschool to underserved children and families. 


In four of the five communities, the PEG program was the first formal early education experience for 
most participating children. 


PEG communities made progress with implementation on all eleven key quality elements in the 
first year of the program. 


The PEG model includes eleven elements perceived to be important drivers of quality such as: 


e Full-day and full-year programming; 

e Class sizes of no more than 20 students; 

e  Teacher-child ratio of no more than 10 students to one teacher; 

e Lead teachers with Bachelor of Arts (BA) degrees; 

e Compensation for lead preschool teachers comparable with that of local public school K-12 teachers; 
e Professional development and coaching for educators; 

e Formal curricula; 

e Formative assessment systems; 

e Family engagement supports; and 

e Comprehensive services for families, with inclusive services for special populations. 


An additional element is the local governance structure including leadership from the participating 
districts and programs. This type of collaborative structure is seen as an important “enabling 
environment” or condition for the development and oversight of the PEG model. 


Although all programs are expected to implement all eleven elements fully, the PEG model is supportive 
of local program development and approaches to implementation. 


Six of the eleven quality elements were fully implemented in all communities in 2015-16. 

These elements included: 

e Classroom operations. All 48 classrooms provided full-day, full-year programming (at least 
8 hours/day, 12 months/year). 

e Class size. All 48 classrooms had enrollments of twenty or fewer students. 

e Staff-child ratio. All 48 classrooms maintained a child-teacher ratio of no more than 10 children to 
one teacher. 

e Curriculum: All of the classrooms used curricula that aligned with the Massachusetts Preschool 
Standards and Guidelines. 

e Assessment. All of the classrooms conducted formative assessments of children using a standardized 
system--Teaching Strategies Gold®. 

e Teacher qualifications. All lead teachers had a minimum of a bachelor’s degree in a relevant field. 

e Teacher compensation. Lead teachers were paid an average of $54,246 (not including fringe 
benefits) in 2015-16, which is in line with what preschool teachers were paid by school districts and 
was higher than the typical teacher salaries in Head Start programs and other child care programs in 
Massachusetts. 
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Five of the eleven PEG quality elements were partially implemented or showed substantial 
variation in implementation across communities and/or centers in 2015-16. 


These elements included: 


e Amount of professional development training. On average, PEG lead teachers reported receiving 
23 hours of professional development (PD) training in 2015-16 across a variety of topics. This ranged 
from as few as 5 hours to as many as 50 hours of professional development. There were substantial 
differences across PEG communities in the amount of professional development that teachers 
reported receiving, with average hours of training ranging from 10 to 29 in the five communities. 


e Level and focus of coaching. Teachers were provided with coaching by LEA staff in four of the five 
PEG communities. The amount of coaching teachers received varied across communities and 
programs. Teachers reported receiving as few as 10 hours of coaching over the year and as many as 
more than 40 hours. The amount of coaching varied across communities. In three communities, more 
than half of the teachers reported receiving at least 20 hours of coaching, while in the fourth 
community, no teachers received 20 or more hours of coaching. 


e Extent of family engagement. Most programs employed a family services coordinator; in four of the 
communities, each program had at least one dedicated staff member. The communication occurred 
through formal parent-teacher conferences and home visits, as well as more informal, ongoing 
discussions between parents and teachers at drop-off and pick-up, and through individual phone calls 
and text messages. In addition, almost two-thirds of centers reported conducting home visits. Most 
PEG centers were involved in helping families with the logistics of kindergarten transition planning 
and providing families with assistance in the kindergarten registration process. 


About half of the center directors reported that over 75 percent of parents participated in family 
engagement activities; other center directors reported lower levels of engagement, including some 
programs in which fewer than 20 percent of parents were reported to be engaged with the program. In 
most programs, some parents volunteered in the classroom, although the frequency of volunteering 
varied across programs, from as often as weekly to once per year. 


e Extent of comprehensive services for families (referrals and direct provision). Nearly all PEG 
center directors reported referring families for additional public services (including services 
addressing health, mental health, and behavioral needs), and about half of the programs provided 
some services directly to families. 


Some parents reported accessing services such as special education, family nutrition, meal guidance, 
and parenting classes through the PEG program. However, some parents reported being unaware of 
what services were offered through PEG and did not receive any services during the year. 


e Strength of local collaboration: Each community formed a collaborative body that represented PEG 
leaders from the LEAs and ELPs, and these collaborations met regularly to develop the PEG program 
design for their community, coordinate program activities across providers, and make decisions about 
implementation goals. Across the five communities, as of spring 2016, PEG leaders were particularly 
positive about the local context in which they conducted their work (the climate for collaboration was 
favorable, and the collaboration was perceived as a leader in the early childhood community), 
believed that the participating organizations saw collaboration as in their best interest, and agreed that 
members of their collaborative had a stake in both process and outcome. There was also widespread 
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agreement that PEG leaders communicated regularly, both formally and informally. PEG 
collaborative members were less positive about the degree to which their collaboration had defined 
and worked toward a common mission. The collaborative members also reported that they did not 
always have sufficient human or financial resources for full implementation of their collaborative 
work. 


Detailed Findings on Teacher and Parent Experiences 


The first-year evaluation included preliminary assessments of PEG teacher attitudes and experiences as 
well as parent experiences with the PEG programs and their home educational supports. 


PEG teachers reported moderate to high satisfaction with being a PEG teacher. 


One of the goals of PEG is to support teachers’ sense of professionalism and self-efficacy through 
professional development, coaching, and compensation. Ultimately, it is expected that teachers who feel 
more positively about their jobs and their own competencies will remain in the program, thereby 
providing the program continuity and stability. 


In 2015-16, most PEG lead teachers felt positively about their job: half reported feeling very satisfied and 
another 44 percent reported feeling somewhat satisfied with their jobs. More than 90 percent of PEG 
teachers expressed a high level of confidence in their ability to make a significant difference in their 
students’ lives and in reaching even the most difficult students. Overall, teachers were highly confident in 
their ability to communicate with parents and guardians. 


PEG parents reported feeling strongly connected to their programs. 


The PEG model assumes that involving families will be beneficial to children’s learning and development 
by building a connection between the home and the preschool program and by supporting parent efforts to 
be more active teachers of their children in the home. By the end of the first year of PEG, more than 

75 percent of surveyed parents reported that they were strongly connected to their child’s PEG classroom. 
Across the five communities, the percentage of “strongly connected” parents ranged from 66 percent to 
90 percent. While nearly all parents (93 percent) also felt well-informed about meetings and special 
school events for parents, somewhat fewer (81 percent) agreed that parent activities were scheduled at 
convenient times. 


PEG parents reported feeling confident about their ability to communicate with their child’s 
teacher, and reported that teachers did a good job of informing them about their child’s progress 
in school. 


Families expressed satisfaction with the quality and intensity of their communications with teachers. 
Nearly all parents indicated that teachers kept them informed them about their child’s progress in school 
(91 percent) and that teachers were interested in their children and cooperative with their families 

(95 percent). Over 90 percent of parents were confident about their ability to communicate with their 
child’s teacher, their ability to communicate effectively with their child about the school day, their 
knowledge of what their child was learning, and their ability to help their children continue their learning 
at home. A slightly lower proportion of parents reported that they had the skills to help out at their child’s 
program (83 percent). 
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The majority of PEG parents reported reading to their child regularly. 


About one-third of parents reported reading to their child every day and another 50 percent of parents 
reported reading to their child a few times per week. Nearly all parents also reported engaging with their 
child every day or a few times per week in activities involving numbers and/or letters. 


Next Steps for PEG 


In the second year of PEG (2016-17), a goal for the program is to increase the level of 
implementation of the key quality elements. 


The focus includes: 


e Continued community collaboration with strengthening of the relationships across early childhood 
systems and expanded joint planning and programming; 

e Continued supports for teachers with more consistent implementation of professional development 
and coaching across all communities and centers; 

e Continued engagement of families and building the connection of all families to the program; and 

e Support for an increasing number of families with access to comprehensive services. 


In the second year of PEG, another goal is to achieve positive outcomes for teachers, classrooms, 
families and children. 


The outcomes include increasing or continuing: 


e High levels of teacher satisfaction and retention; 

e High levels of classroom instructional quality; 

e High levels of parent connectedness to the program and home support for children’s development and 
learning; and 

e Age-appropriate levels of skill development for children in all domains so that children enter 
kindergarten ready to learn. 


In the second year of the PEG evaluation (2016-17), ongoing attention will be paid to the implementation 
of the different components of PEG, including the level of classroom quality. The evaluation will also 
generate evidence about the extent to which the PEG programs are providing high-quality environments 
relative to other Massachusetts programs, the progress of PEG children’s development over the course of 
their preschool year, and the impact of PEG on children’s school readiness and parent behaviors and 
supports. 
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Research shows that early education can be a cost-effective investment that improves the long-term 
school achievement and engagement outcomes for young children, particularly those at risk of adverse 
outcomes due to growing up in poverty (Campbell, Ramey, Pungello, Sparling, & Miller-Johnson, 2002; 
Schweinhart et al., 2005; Yoshikawa et al., 2013). In the last decade, in addition to investments in early 
childhood education by the federal government, many states and municipalities have also provided 
funding to expand the availability of center-based early childhood education, with promising results 
(Gormley, Gayer, Philips, & Dawson, 2005; Weiland & Yoshikawa, 2013; Wong, Cook, Barnett, & Jung, 
2007). Massachusetts has long been at the forefront of efforts to support early learning, and the 
Massachusetts Department of Early Education and Care (EEC) has been highly involved in providing an 
array of supports to help ensure that children, especially those with the highest needs, arrive in 
kindergarten with the skills and abilities necessary to succeed. 


An ongoing investment in preschool education is the Preschool Development Grant Program, a 
discretionary federal grant program that is jointly administered by the U.S. Department of Education and 
the U.S. Department of Health and Human Services. Preschool Development Grants support states to 

(1) build or enhance their infrastructure to provide high-quality preschool programs (referred to as 
Preschool Development Grants) and (2) expand high-quality preschool programs in high-need 
communities (referred to as Preschool Expansion Grants). The program currently supports 18 grantees 
(13 Expansion and 5 Development) nationwide. Each state grantee uses their funds in a unique way to 
extend and expand high-quality preschool. 


In late 2014, EEC was awarded a federal Preschool Expansion Grant (referred to in this report as the 
Massachusetts PEG program) to expand high-quality early education to four-year-old children whose 
families earn under 200 percent of the federal poverty line. The PEG program provides the 
Commonwealth with a unique opportunity to increase access to high quality preschool through a mixed 
delivery system' and has allowed EEC to pilot a model that, if successful, might be replicated more 
widely across the Commonwealth. In 2016, the Massachusetts Legislature budgeted $500,000 for 
planning grants to support additional communities in identifying how they might expand preschool 
opportunities for both three- and four-year olds, following the model being piloted in the PEG 
communities. 


EEC has completed its first year (2015-16) of overseeing the PEG program, which provided resources to 
48 classrooms in five underserved communities across Massachusetts. In each community, lead education 
agencies (LEAs) were granted the funds and subcontracted with early learning providers (ELPs) for the 
direct services to preschool children and families. Participating LEAs and ELPs are following a model 
that is intended to deliver the ingredients and supports that research has shown can lead to improved child 
outcomes. 


As part of the PEG program, EEC has invested in a rigorous external evaluation. The PEG evaluation 
represents an important opportunity for EEC to study a potential statewide approach for supporting high 
quality state-funded preschool. The PEG evaluation is being conducted by an independent research firm, 


' The mixed delivery system refers to an early education system that includes child care centers, family child care 


educators, public preschool programs, and Head Start and Early Head Start programs. 
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Abt Associates Inc., in partnership with the University of Massachusetts Boston. The evaluation has four 
main components including: 


An implementation study of the PEG quality components in PEG communities and programs; 
A longitudinal study of outcomes for PEG children and families; 

An impact study of effects on PEG children and families; and a 

A cost study. 


The impact study will begin with the cohort of children entering preschool in the fall of 2016 and 
continue with a second cohort of children entering in the fall of 2017. 


This report describes the first year of implementation of the PEG program in Massachusetts and provides 
an initial assessment of the extent to which PEG programs are meeting the objective of providing high- 
quality preschool to underserved children. The report is organized into the following chapters: 


Overview of the Massachusetts PEG program; 
Overview of the Massachusetts PEG evaluation; 
Implementation of the PEG model in the first year: 
— Collaboration and leadership; 
— Teacher supports; 
— Curriculum and learning including observed classroom quality; 
— Family engagement activities; and 
Comprehensive services for families; 
Status of PEG children’s development at the end of the 2015—16 preschool year; and 
Overall assessment of the first year (2015-16) of the PEG program’s implementation progress and 
implications for future years. 
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2. Overview of the Massachusetts PEG Program 


This chapter provides an overview of the Massachusetts PEG program structure, expectations for 
participating programs, research basis for the state-level program model, and characteristics of 
participating children. 


2.1. Structure of the PEG Program 


Massachusetts is using its PEG grant to fund five high-need communities—Boston, Holyoke, Lawrence, 
Lowell, and Springfield—to expand full-day, full-year preschool for four-year-old children through 
public-private partnerships between the LEA and licensed ELPs. 


PEG funds are allocated to communities using a block grant model that is based on a per-student 
allocation that started with the Commonwealth’s Chapter 70 education foundation grant amount and was 
prorated to account for expanded hours per day (eight versus six hours per day). The funding level for 
each community has a floor to ensure a minimum investment in the smallest community (Holyoke) and a 
ceiling, along with an enhanced amount based on the higher cost of living, for the largest community 
(Boston).’ 


The PEG grant is implemented using a mixed delivery system; the five LEAs are granted the funds and 
then subcontract with ELPs for the direct services provided to children and families in the community. As 
Exhibit 2.1 shows, in each community the LEA subcontracted with several participating ELPs, including 
a Head Start agency and one or more community-based agencies. 


In 2015-16, some ELPs operated PEG classrooms in a single location (referred to as a center in this 
report) whereas others operated PEG classrooms across multiple locations. Prior to the PEG grant, all 
participating ELPs had experience administering preschool classrooms and managing the licensing of 
facility space. 


EEC staff actively collaborated with the designated LEAs and ELPs in the planning and early 
implementation process, especially in the local planning for professional development activities during 
the first year of implementation (2015-16). The designated ELPs worked together with the LEA around 
the selection and implementation of curriculum, coordination and provision of comprehensive services, 
family engagement supports, and inclusive services for special populations, as well as joint professional 
development. 


Beginning in September 2015, ELPs began to operate 48 PEG classrooms across 24 centers, although full 
enrollment was not required until December 2015. In 2015—16, 18 of the 24 PEG centers included a 
single PEG classroom, and the other six centers each operated between two and eight PEG classrooms. 
Most PEG classrooms were managed by a single ELP, though two communities established new centers 
with PEG classrooms that were overseen by more than one ELP. In four of the five communities (all 
except Boston), the PEG classrooms were new classrooms. In Boston, PEG funding was used to support 


As described in chapter 4, Boston also made the decision to blend PEG funding with child care subsidy funding. 
An additional allocation per student was identified as a supplement to the child care subsidy funds, at an amount 
that accounted for the normal parent fee (which was waived in PEG) and the costs of the quality enhancements 
required by PEG. 
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existing preschool classrooms that implemented the PEG operating schedule (i.e., extending the programs 
to offer full-day, full year care) and all elements of the PEG instructional model. 


Exhibit 2.1: Number of PEG Participating Organizations and Classrooms by Community, 2015-16 


# of PEG 
Local Education Agency # of ELPs # of PEG Centers Classrooms 
Boston Public Schools 8 12 15 
Springfield Public Schools 3 52 11 
Holyoke Public Schools 2 4 
Lawrence Public Schools 2 2 10 
Lowell Public Schools 2 1b 8 
Overall 16¢ 24 48 


aln Springfield, three ELPs jointly operated one of the five centers. 
>In Lowell, two ELPs jointly operated one center 
cOne ELP operated PEG classrooms in two communities (Springfield and Holyoke). 


To be eligible for PEG, children in each of the five communities were required to meet two criteria: 


e Child must have reached his/her fourth birthday by the beginning of their preschool year and be 
eligible for kindergarten in the following September, and 
e Family income must be below 200 percent of the federal poverty line. 


Four of the five PEG communities (all except Boston) targeted and primarily served children who had 
never been enrolled in licensed early education (including both center-based programs and licensed 
family child care homes) in the prior year. In Boston, many of the children in the PEG classrooms were 
already served by licensed care settings prior to their PEG preschool experience. 


2.2 PEG Program Model and Rationale 


Although the Massachusetts PEG model is supportive of local program development, each PEG program 
is expected to provide the following eleven elements, which are perceived to be important components of 
quality early education: 


e Collaborative local governance structure to oversee implementation and work on systems 
coordination for all children in the community; 

e Full-day, full-year programming (at least 8 hours/day, 12 months/year); 

e Maximum class size of 20; 

e Maximum child-teacher ratio of 10:1; 

e Curricula aligned with the Massachusetts Preschool Standards and Guidelines (although the specific 
curriculum may vary by grantee); 

e Use of Teaching Strategies Gold® (Heroman, Berke, & Bickart, 2010) as a formative assessment 
tool; 

e One educator in each classroom with a bachelor’s degree in a relevant field; 

e Salaries for all educators commensurate with those of comparable positions in the public schools 
within the respective community; 

e Professional development training and coaching for teaching staff, and other supports for planning 
and implementation of curriculum, in collaboration with the LEA; 
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e Family engagement activities, including support for kindergarten transition and information and 
resources about child development; 
e Comprehensive services including: 

— Services addressing health, mental health, and behavioral needs for all families, 

— Inclusion of students receiving special education services as well as children requiring other 
supports (e.g., children without permanent homes, dual language learners, refugee or immigrant 
families), and 

— Efforts to build links with services for children from birth to age 3 as well as support for the 
transition to kindergarten and connections with public elementary schools. 


By the end of the grant period (2018-19), PEG centers are also expected to attain the highest rating 
(Level 4) in the QRIS or Level 3 and National Association for the Education of Young Children 
(NAEYC) accreditation. 


Within the PEG model framework, LEAs and ELPs are provided with flexibility regarding the specific 
approaches they take to implement each element mentioned above. Some of the ways in which PEG 
communities and/or programs differ include (but are not limited to): 


e Location of services (some ELPs co-locate all PEG classrooms within one center, while others 
provide services across various centers in the community); 

e Approach to collaboration with other community providers; 

e Size of the additional compensation to PEG teachers; 

e Curriculum/a used in the classrooms; 

e Focus and design of professional development and coaching supports offered; and 

e Focus of family engagement efforts and the provision of comprehensive services. 


The requirements guiding the PEG program model are intended to ensure the delivery of high quality 
ingredients and supports that research has shown will improve child outcomes, especially for children at 
risk for academic failure. The links between the required ingredients and both short- and long-term 
outcomes are shown in the PEG program theory of change (Exhibit 2.2). 


The PEG model (as conceptualized) is intended to achieve a high level of quality in instructional and 
emotional supportiveness, classroom organization, and learning resources. It also includes goals beyond 
those pertaining to program quality and outcomes for educators, parents, and children. For example, the 
model has an explicit focus on systems-building, as represented in the public-private and cross-agency 
collaboration that is expected to be developed among the key stakeholders in the early education system 
in each community. 


The teacher-focused supports that PEG LEAs and ELPs are providing are expected to lead to greater job 
satisfaction, improved self-efficacy, and the increased ability to recruit and retain high-quality educators, 
which are believed to lead to sustained improvements in classroom quality and ultimately child outcomes. 
The family engagement activities and comprehensive services are expected to lead to improved parent 
and child outcomes, including improved family stability, child behavior and attendance, and reduction of 
the need for services in elementary school. 
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Exhibit 2.2: Theory of Change for Massachusetts Preschool Expansion Grant 


Program (ELP)-Level Inputs: 
Implementation 


Program Outcomes 
Child and Family 


State-Level Inputs/Activities 


LEA-Level Inputs/Activities intormediaie 


Child outcomes 
* End of preschool and early 


* Curriculum and instruction aligned Teacher outcomes 
to state standards Stronger instructional skills 


Provide support, coordination, and Manage and oversee PEG 
technical assistance to LEAs subgrant 


Collaborate and coordinate with 
other state agencies around early 
education policy 


Coordinate collaboration among * Formative assessments to guide Increased engagement with elementary school child outcomes 
child- and family-serving agencies families Reduced turnover * Socio-emotional skills 
Lead coaching for PEG educators Higher sense of efficacy and job * Promotion 
Professional development and 
IDs for children coaching for PEG teachers Increased knowledge of high- * Attendance 
Lead and/or help with recruitment quality instruction, especially for 
including: * Satisfaction with program 
Parent education and home Classroom outcomes engagement activities 
Parent involvement in program programming success in kindergarten and 
and classroom activities » Effective instructional strategies beyond 


directors of ELPs and other local instruction, communication with professional development * Early academic skills 
Responsible for obtaining state commitment/satisfaction * Reduced suspension 
of PEG families and children Family engagement activities high-needs subgroups Family outcomes 
supports for learning * High-quality instruction and * Increased ability to support child 
Communication with families 


for diverse learners 

about child progress 

System outcomes 

* Increased access to high-quality 
early education for underserved 
populations in the community 

* Stronger, more connected early 
childhood education system 


Link families/children with 
comprehensive wrap-around 
services for development and 
family needs 

Provide or coordinate services for 


Increased knowledge of ability to 
support child’s learning in home 
Increased access to needed 
services for family members and 
children 


special needs children 
Kindergarten transition supports 
for PEG families 


Alignment of programming with state 
QRIS standards 


Participation in collaboration and 
coordination activities with other 


Note: This logic model was developed by the study team to 
help guide the evaluation. 


early education providers in 
community, including LEA 
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2.3 Children Enrolled in PEG 


Research shows that children from low-income households are more likely to enter school lacking some 
key skills for school success and more likely to demonstrate lower academic performance in school. The 
children enrolled in PEG in the 2015-16 program year came from low-income families; the mean PEG 
family income reported as part of the enrollment process was $18,466 dollars per year. Families were 
required to earn below 200 percent of the federal poverty level to be eligible for PEG. In fact, the majority 
of families earned well below this threshold, as 66 percent of families reported incomes below 

100 percent of the 2016 federal poverty level for a family of four ($24,300). 


In addition to growing up in a low-income household, the majority of the PEG children were from racial 
and/or ethnic minority groups, and over one-third (39 percent) lived in households where English was not 
the primary language spoken (see Exhibit 2.3). In addition, a small proportion of children (3.6 percent) 
were reported by programs as having Individualized Education Programs (IEPs), a plan developed for 
each public school child eligible for special education to guide his or her services. 


As Exhibit 2.3 shows, the racial/ethnic makeup of the PEG classrooms varied by community. Overall, the 
majority of children served in PEG classrooms in 2015—16 were Hispanic (58 percent), and the proportion 
ranged from 34 percent of PEG children in Boston to 100 percent of PEG children in Holyoke. About 
one-quarter of children (27 percent) served were Black, ranging from two percent in Lawrence to more 
than 55 percent in Boston. Almost one-fifth of children served in PEG classrooms in Lowell were Asian- 
American, while there were very few Asian-American children in the other communities (one percent or 
less). Spanish was the primary language of 29 percent of children across the five PEG communities, 
ranging from 17 percent in Springfield to 73 percent of children in Lawrence. Notably, 26 percent of 
students in Lowell reported speaking a primary language other than English or Spanish (mostly Khmer 
and Portuguese), reflecting the diversity of that community. 


Exhibit 2.3: Race/Ethnicity and Home Language, Overall PEG and by Community 


Number and Percentage of Children 
Boston Holyoke Lawrence 


Overall PEG 


Lowell Springfield 


Race/Ethnicity 


Caucasian—non-Hispanic 59 T% 14 5% 0 0% 3 2% 34 22% 8 4% 
Hispanic 486 58% | 94 34% 66 100% 136 95% 56 35% 134 80% 
Black 223 27% 
Asian-American 35 4% 
Two or more races 20 2% 


American Native 1 0.1% 


Other 13 1.6% 
Primary Home Language 
English 516 60% 74% 38 27% 


Spanish 249 29% | 56 20% 23 35% 104 73% 31 20% 32 17% 
Haitian Creole 19) 2% | 7 6% 2 2% 1 1% 0 0% 0 0% 
Other 71 8% | 26 9% $2 2% 0 0% 41 26% 3 2% 


aOther common languages included Portuguese, Khmer, and Chinese; the most common other language was Khmer. 
Notes: Data obtained from EEC data for all 48 PEG classrooms during spring/summer 2016. 
Percentages may not add up to 100 because numbers are rounded to the nearest whole, except when they are between 0 and 1%. 
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Half of PEG parents reported that they had a high school degree or GED only, and 14 percent had 
Associate’s degrees (14 percent), vocational or technical certificates (13 percent), or bachelor’s degrees 
(9 percent). Most PEG households (72 percent) included at least two adults with full-time jobs. Sixteen 
percent of households had one adult with a full-time job, and the remaining households (12 percent) had 
no adult with a full-time job. 


The next chapter of the report provides an overview of the PEG evaluation, and is followed by an in- 
depth examination of the implementation of each component of the PEG program model during 2015-16. 
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The multi-year PEG evaluation includes four main components: an implementation study of quality 
components in PEG programs, a longitudinal study of outcomes for PEG children and families, an impact 
study of effects on children and families, and a cost study. Beginning in Year 2 (2016-17) of the 
evaluation, the study team will begin the longitudinal study of outcomes for PEG children and families 
and the impact study of the effects of PEG on children and families. A cost effectiveness study will be 
conducted beginning in Year 3. 


This chapter describes the following features of the Year 1 (2015-16) evaluation: research questions; data 
sources, sample description and measures; and approach to data analysis. 


3.1 Research Questions 


Year | of the PEG evaluation focused on the implementation study, including collaboration, teacher 
supports and classroom quality, and comprehensive services and family engagement activities. The 
Year | evaluation was designed to answer the following research questions: 


e What types of collaborative activities/planning/decision-making took place among participating ELPs 
and LEAs? To what extent were collaborative activities effective? What challenges were 
encountered? 

e What professional supports were provided to PEG teachers? What was the sense of self-efficacy and 
job satisfaction among PEG teachers? 

e To what extent were PEG classrooms providing children with high-quality learning experiences? 

— What curricula and assessment systems were implemented in PEG classrooms? 
— How much time did PEG children spend in different types of activities? 
— What was the nature of observed classroom quality in PEG classrooms? 

e How did PEG programs engage with families? How satisfied were PEG families with these activities? 

e To what extent did PEG program make referrals to and provide comprehensive services for PEG 
families? How well-served did families feel about the services offered and received? 

e What capabilities across different developmental domains did PEG children show at the end of their 
preschool year? 


3.2 Data Sources, Sample Description, and Measures 


The study team collected primary data during Year 1 through surveys, interviews, focus groups, 
classroom observations, and child assessments (Exhibit 3.1). 
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Exhibit 3.1: Overview of Data Sources and Instruments, by Focus Area 


Focus Area Sources Instruments 
Implementation of PEG Surveys, Interviews, and Teacher Survey; Parent Survey; Center Director Interviews; 
quality elements Focus Groups. LEA/ELP Leader Interviews; Coach Interviews; Teacher Focus 
Group; Parent Focus Group; Wilder Collaboration Inventory 
Classroom quality Standardized observation 3 classroom quality measures administered over 2 days in each 
measures (described on later classroom 
slide) 1 time-sampled measure of activities and child-teacher groupings 
Children’s pre- Direct assessments 3 standardized measures of early literacy and early math and 
academic skills (described on later slide) language development, administered individually to 8 randomly- 


selected PEG children/classroom in late spring 


3.2.1. Surveys, Interviews and Focus Groups 


Exhibit 3.2 provides additional details about the samples and response rates for the surveys, interviews, 
and focus groups conducted during Year 1. 


Exhibit 3.2: Year 1 Survey, Interview, and Focus Group Sample and Response Rates 


Sample Response Rate 
Surveys 
Teacher survey All lead teachers and e Survey sent to 118 teachers (52 lead teachers and 66 assistant 
assistant teachers with teachers) 


available contact information « Completed surveys obtained from 39 lead teachers (75 percent 
response rate) representing 35 of the 48 PEG classrooms and 
33 assistant teachers (50 percent response rate) representing 27 of the 
48 PEG classrooms 


Parent survey All parents of PEG childrenin ¢ Survey sent to 760 parents 


48 PEG classrooms e Completed surveys from 336 parents/guardians (44 percent response 
rate) 

Wilder All members of PEG e Inventory sent to 59 identified leadership team members across 
Collaboration leadership team in each 5 communities 
Inventory community e Completed inventories from 49 respondents (83 percent response rate) 
Center director All PEG center directors e Interviews conducted with 28 center directors across 24 ELPs (includes 
interview teacher directors) (100 percent response rate) 
LEA/ELP leader AllLEAcoordinators and ELP Interviewed 5 LEA coordinators 
interview leaders in each community e Interviewed 10 ELP leaders across 16 agencies 
Coach interview All PEG coaches e Interviews conducted with 10 coaches in 4 communities where PEG 


coaching was provided 


Teacher focus One focus group in each e 8 teacher focus groups conducted, at least 1 per community 
group community, inviting all lead e 40 teachers participated (primarily lead teachers) 
teachers from all PEG 
classrooms 
Parent focus Multiple focus groupsineach 11 focus groups conducted (at least 1 per community and up to 3 in 
group community, inviting all 2 communities) 


parents from 12 PEG centers 70 parents participated 
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3.2.2 Classroom Observations 


The study team conducted two-day classroom observations in February and March 2016 in all 48 PEG 
classrooms. Trained and reliable observers were in the classroom from the start of the school day until 
children began naptime (generally 8 a.m.—1 p.m.), and used four structured observation measures (two on 
each day). 


The structured observation measures included: 


e Classroom Assessment Scoring System for Pre-K (CLASS; Pianta, La Paro, & Hamre, 2008). The 
CLASS measures overall instructional quality with a focus on interactions among teachers and 
students in the classroom. Each item score ranges from 1 to 7. A score of 1—2 is described as “low” 
quality in that aspect of teacher-child interaction. Scores of 3—5 are described as “moderate,” and 
scores of 6—7 are described as “high” quality. 

e Early Language and Literacy Classroom Observation Pre-K (ELLCO; Smith, Brady, & 
Anastasopoulos, 2008). The ELLCO captures more in-depth information on the quality of support for 
language and literacy development, including support for diversity of languages, abilities, and 
cultures. Each item score ranges from 1 to 5, with the highest scores described as “exemplary.” 

e Classroom Observation of Early Mathematics—Environment and Teaching, version 3 
(COEMET; Sarama & Clements, 2007). The COEMET focuses on the quality and quantity of 
mathematics instruction and measures the richness of the math environment in the classroom. The 
overall instructional environment is rated (1—5 scale) along with a count and rating (1—5 scale) of 
specific math activities (SMAs) and a count of mini math activities (“minis’’)— those that are brief 
and/or do not involve teachers or assistants but which nevertheless include mathematics content. 

e Observation Measures of Language and Literacy Instruction in Early Childhood Education 
Classrooms—Snapshot of Classroom Activities (QMLIT; Goodson, Layzer, Smith, & Rimdzius, 
2005). The Snapshot portion of the OMLIT measures classroom configurations and activities for each 
child and staff member in the classroom. The Snapshot provides a perspective on what a classroom 
looks like, based on how children spend their time—the activities they participate in and whom they 
are interacting with (i.e., number of other children, staff). In the Snapshot, 15 types of activities are 
coded, for example, time spent in reading, math, or science, and, within each activity that is 
occurring, the size of the child instructional grouping (whole group, small group, or individual 
activity) and how much of the time children are with adults. 


The CLASS and the ELLCO were administered simultaneously during the first day, and the COEMET 
and the Snapshot were administered together during the second day. 


3.2.3. Child Assessments 


Toward the end of the first year of PEG (June—July 2016), the study team conducted individual child 
assessments on selected PEG children using standardized measures of early math, early literacy, and 
vocabulary. To select the sample, the evaluation team first sent parent permission forms home to 760 PEG 
children, and 543 signed permission forms were returned. 


The study team selected a random sample of up to eight children per classrooms to directly assess. The 
sample selected reflected the primary languages of the children spoken at home in each classroom; that is, 
if 20 percent of the children in a classroom had a primary language other than English at home, the 
sample selected included 20 percent of the children in this group, to the extent that there were sufficient 
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children with permission to do so. In classrooms with less than eight children with parent permission, the 
study team directly assessed all children with permission. Assessments were conducted with 331 children 
across the 48 PEG classrooms. 


The standardized measures used to assess the selected children included: 


e Early Math: Woodcock-Johnson III Tests of Cognitive Abilities: Applied Problems Subtest 
(Woodcock, McGrew, & Mather, 2001). The Applied Problems subtest measures the ability to count 
and solve problems related to numeracy and space. The child hears a story problem and is asked to 
recognize the mathematical procedure that should be used and to perform the appropriate calculation. 

e Early Literacy: Woodcock-Johnson III Tests of Cognitive Abilities: Letter-Word Identification 
Subtest (Woodcock, McGrew, & Mather, 2001). The Letter-Word Identification subtest measures 
early literacy skills. The child is asked to identify individual letters and read individual words of 
increasing difficulty. 

e Vocabulary Comprehension: Expressive One-Word Picture Vocabulary Test, Version 4 (EOWPVT; 
Martin & Brownell, 2010). The EOWPVT measures vocabulary knowledge based on the child’s 
ability to name an object/action/concept presented in a picture. The pictures represent objects or 
concepts that children usually have experience with through home, school, or the media. 


Children identified as living in homes where Spanish was the primary language were first screened on 
their English language proficiency at the time of the assessments. The preschool version of the Language 
Assessment for Early Learners (preLAS; Duncan & De Avila, 1998; preLAS 2000, CTB/McGraw-Hill), a 
structured English language proficiency screener, was administered to these children, and children who 
failed to pass the screener were administered the Spanish language versions of the early math and early 
literacy tests and a bilingual version of the vocabulary test. The Spanish/bilingual versions of the 
assessment measures included: 


e Early Math: Bateria IIT Woodcock-Munoz Problemas aplicados (Munoz-Sandoval, Woodcock, 
McGrew, & Mather, 2005); 

e Early Literacy: Bateria II] Woodcock-Munoz Identificacion de letras y palabras (Munoz-Sandoval, 
Woodcock, McGrew, & Mather, 2005); and 

e Vocabulary Comprehension: Expressive One Word Picture Vocabulary Test, Version 4, Spanish- 
Bilingual Edition (EOWPVT; Martin, 2012). 


In addition to the direct assessments of academic achievement, the study team obtained Teaching 
Strategies (TS) GOLD® teacher ratings from EEC on 606 PEG children from 40 of the 48 PEG 
classrooms. The ratings were completed in the spring or summer of 2016. The evaluation focused on the 
three Social-Emotional Objectives from the TS GOLD which included ratings of nine individual skills: 


e Objective 1: Regulates own emotions and behaviors 
— Skill 1: Manages feelings 
— Skill 2: Follows limits and expectations 
— Skill 3: Takes care of own needs appropriately 
e Objective 2: Establishes and sustains positive relationships 
— Skill 4: Forms relationships with adults 
— Skill 5: Responds to emotional cues 
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— Skill 6: Interacts with peers 
— Skill 7: Makes friends 
e Objective 3: Participates cooperatively and constructively in group situations 
— Skill 8: Balances needs and rights of self and others 
— Skill 9: Solves social problems. 


3.2.4 Other Available Data 


The study team also obtained additional available program data from EEC including demographic 
characteristics about PEG teachers, parents, and children. 


3.3. Approach to Data Analysis 


The study team conducted a variety of descriptive data analyses to provide an in-depth overview of the 
implementation and early child outcomes of the PEG program in its first year. To examine the level of 
implementation of each component of the PEG model (1.e., collaboration, professional development, 
coaching, family engagement, and comprehensive services), the study team analyzed survey, interview, 
and focus group data from multiple stakeholders including agency leadership, district staff, center 
directors, coaches, teachers, and parents. The evaluation team also examined whether there were 
differences in program implementation by community. 


To examine characteristics of members of the PEG community, the study team examined measures from 
teachers, classrooms, parents, and children that included teacher and parent surveys, classroom 
observations, and child assessments. Parent measures included program connectedness and home 
activities, while teacher measures included self-efficacy and job satisfaction, classroom measures 
included the quality of teacher-child interactions and how child time was spent, and child measures 
included multiple domains of early childhood development (early math, early literacy, and vocabulary 
comprehension). 
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One of PEG’s goals is to improve collaboration and foster leadership within the early education system in 
each of the participating communities. In service of this goal, LEAs and ELPs in each community formed 
collaborations to jointly plan for and implement the PEG model in their communities. Communities 
expect that adjustments will be made to the local models as the program develops. 


This chapter starts with a brief overview of the collaboration models that were developed in the 
communities and the collaborative activities that took place through the first year of PEG implementation 
(2015-16). The chapter also describes some specific successes of and challenges to collaboration that 
local PEG leadership reported having experienced through in the first year. The study team conducted 
structured interviews with key PEG leaders in each community, including representatives from each PEG 
LEA and PEG ELP. Finally, the findings from a standardized collaboration inventory are summarized as 
a broader reflection on strengths and weaknesses of the collaborative efforts. Data on PEG collaboration 
were collected from local PEG leadership teams near the end of the first year of implementation (during 
spring 2016); 48 PEG leaders completed the Wilder Collaboration Factors Inventory (Mattessich, 
Murray-Close, & Monsey 2001), including LEA staff and at least one representative from each ELP in 
four of the five communities and all but one ELP in the other community. 


Throughout this chapter, members of the collaborative group in each community are referred to as the 
“PEG leadership” or “PEG leaders.” PEG leadership primarily consisted of LEA PEG coordinators and/or 
other involved district staff, ELP leaders, and in some cases PEG center directors. 


4.1 Summary of Findings 


e Ineach community, a governing body was formed that represented PEG leaders from the LEAs and 
provider organizations and made decisions related to the design and management of the PEG funded 
programs. The communities took different approaches to this collaborative work. All of the 
communities coordinated some programmatic activities across providers and engaged in collaborative 
decision-making. 

e ELP leadership and center directors reported both successes and challenges related to working 
collaboratively to implement PEG and carry out a shared vision for serving all children and families 
in their communities. The specific successes and challenges reported varied by community. 

e Across the five communities, PEG leaders were particularly positive about the local context in which 
they conducted their work, believed that the participating organizations saw collaboration as in their 
best interest, and agreed that members of their collaborative had a stake in both process and outcome. 
There was also widespread agreement that PEG leaders in the collaboration communicated regularly, 
both formally and informally. 

e PEG leaders were less positive about the degree to which their collaboration had defined and was 
working toward a common mission. Nor did they feel that they always had the necessary human and 
financial resources available for their work. 

e The PEG community collaborations are expected to develop and strengthen over time. The evaluation 
will continue to measure the progress and strength of local collaboration in future years. 
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4.3. Collaboration Models 


The PEG communities developed distinct collaborative models for implementing PEG, informed by their 
community’s unique history and context. Each community’s model is described briefly below. 


4.3.1. Boston 


In Boston, the PEG collaboration built off a prior collaboration (called Boston KIDS) between the Boston 
Public Schools (BPS) and some of the PEG-funded ELPs. The PEG collaboration began with three ELPs 
and BPS. The LEA and ELPs worked in the winter and summer (2014—15) to design components of the 
grant, including approaches to family engagement and comprehensive services, as well as birth to grade 
three alignment activities. As part of this planning process, the ELPs determined that there was not 
enough unmet need in Boston to justify the eligibility requirement that PEG children had no prior 
preschool. The Boston ELPs instead decided to blend PEG funding with subsidy funds from EEC and 
focus on improving the quality and extending hours of programs and classrooms that currently existed. 
Once the three ELPs identified the number of children they could serve, BPS conducted a follow up 
procurement process to identify additional ELPs to fund with the money that remained; these additional 
ELPs were all drawn from the pool that had previously worked with BPS in implementing the KIDS 
model. 


BPS worked with the ELPs to implement the same curricula currently being used in BPS preschool 
classrooms. BPS provided curricular materials, professional development, and coaching to PEG teachers. 
In two cases, BPS worked in collaboration with additional coaches hired by the ELPs to provide ongoing 
support between BPS coach visits. Each ELP managed the family supports and comprehensive services 
relatively independently, although monthly director meetings allowed for sharing of best practices. 
Subcommittees of center directors were also formed to review current family engagement and 
comprehensive service approaches and develop an approach to be implemented across PEG programs in 
the community. Executive leadership of each ELP also met quarterly to discuss implementation 
challenges. 


4.3.2 Holyoke 


In Holyoke, two ELPs oversaw four PEG classrooms located in Holyoke Public School (HPS) buildings, 
with each ELP responsible for one classroom in each school. HPS provided coaching and coordinated a 
larger initiative focused on building early literacy community-wide (Holyoke Early Literacy Initiative or 
HELI). PEG classrooms are a key component of the larger plans for the community. PEG teachers 
participated in professional learning communities with public preschool and kindergarten teachers. Each 
ELP managed the family engagement and comprehensive service efforts independently and supplemented 
the professional development provided by HPS. The PEG leadership created a steering committee that 
met monthly. PEG leaders also participated in workshops organized by HELI. 


4.3.3. Lawrence 


Two ELPs opted to each start new programs that each ran independently in Lawrence. Lawrence Public 
Schools (LPS) managed enrollment for the PEG classrooms and aimed to identify ways to increase 

alignment with public school kindergarten classrooms, recognizing that each public school in Lawrence 
operates fairly autonomously and there is no one clear model or uniform curriculum for kindergarten in 
the community. During the course of the first year of PEG, the two ELPs began to consider possibilities 
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for greater alignment across their PEG programs and decided to use the same curriculum starting in the 
second year, a change that may also lead to coordinated professional development and coaching. 


4.3.4 Lowell 


Lowell Public Schools (LPS) decided to open one new early childhood center jointly run by two PEG- 
funded ELPs. Although the two ELPs maintain separate licenses for the classrooms they run, the program 
was viewed as one and decisions were highly collaborative across the ELPs. Lowell Public Schools 
provided coaching and coordinated supports with other district departments as needed. In Lowell, a 
management committee met monthly. The committee formed subcommittees as needed to address issues 
that arose, such as choosing a curriculum, coordinating outreach, and strategizing about the need for 
transportation. An executive committee provided updates to the executive management of each ELP 
quarterly. 


4.3.5 Springfield 


During the planning of the PEG grant, Springfield Public Schools (SPS) purchased a building to serve as 
an early childhood center, where PEG classrooms run by three ELPs were co-located with other SPS 
preschool and Early Head Start classrooms. To address the lack of transportation resources, each ELP 
also opened classrooms located within one or two of their other existing sites in the city. All PEG 
classrooms used the same curriculum used by public school preschool classrooms, and SPS provides 
professional development and coaching focused on the curriculum. Each ELP manages the family 
engagement supports and comprehensive services provided to children and families in their classrooms, 
although monthly management meetings for all ELPs support efforts to align these supports. SPS has also 
funded an occupational therapist, a speech pathologist and a behavioral specialist to consult with PEG 
teachers and provide additional comprehensive service supports. Subcommittees were formed to address 
particular areas of focus, such as family engagement. 


4.4 Collaboration Structure 


Each PEG community was required to set up collaborative structures for communication and 
collaboration between the public and community-based early education and care programs. These 
collaborations were expected to support program design, coordination of program activities and funding, 
and coordinated decision-making among the LEA and participating ELPs. In all PEG communities, the 
primary mechanism for this collaboration was the creation of a steering committee with representation 
from the LEA and each ELP. In some communities, multiple committees were formed to ensure 
communication at different levels of leadership, such as subcommittees of executive management and of 
center directors. Also, subcommittees were sometimes formed to manage the planning of particular 
programmatic requirements of the grant, such as family engagement supports, curriculum choices, and 
professional development activities. 


The community steering committees began meeting as soon as the grant award was announced (late 2014) 
and met through the spring and summer of 2015 to plan the program. Meetings continued through the first 
year of the PEG program operation (2015-16), typically held monthly. Ongoing communication among 
PEG leaders was often handled through email. As part of the collaborative work, PEG leaders determined 
which of the PEG programmatic requirements would be managed independently by each ELP and which 
would be coordinated across ELPs. While joint professional development within each community was a 
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requirement of the grant, communities were free to coordinate as many of the other grant requirements as 
they wished. 


As already described, the LEA in each community served as the lead agent for the grant and was 
responsible for managing the funds and monitoring the fund use and the programmatic requirements of 
the grant. Public school preschool classrooms within the participating LEAs did not serve children; the 
PEG classrooms were run by the community partners. Instead, each LEA identified particular supportive 
services, such as professional development, that they would provide. In this role, each LEA hired one or 
two people to serve as the coordinator of the PEG grant and to manage the collaboration among partners. 


4.5 Collaboration Experiences: Successes and Challenges 


Local district and program leadership members reported a range of specific successes and challenges 
related to collaboration during 2015-16. 


4.5.1. Successes of PEG Community Governance Structure 


Overall, PEG leaders from each community reported through interviews that the collaborative structure 

was beneficial to implementing the PEG components and strategizing more systematically about how to 
serve and provide high-quality early education to all children in the community. Collaborative members 
found it helpful to share successes and challenges and learn from one another. 


PEG leaders from four of the five communities also found it helpful to collaborate to provide joint 
professional development and coaching. Leadership from three communities noted that it was helpful to 
work together (across ELPs and sectors) to conceptualize and clearly define a community approach to 
implementing PEG components that were new to some or all of the participating programs. The new 
components most referenced were comprehensive services and transition to kindergarten activities. 
Furthermore, PEG leaders noted that the regular meetings and communications with other programs 
resulted in increased knowledge of community activities and resources that could be shared with their 
families. 


In some communities, leaders noted that the collaborative process and regular communications resulted in 
stronger relationships between the ELPs and LEAs and the opportunity to break down some of the 
separation between different sectors in terms of policy and practices and start to better align the sectors 
into one coherent system. ELP leaders reported gaining a better understanding of the LEA approach to 
early education and programming, and vice versa. In addition, some PEG leaders noted that it was helpful 
to have the Head Start agency perspective represented given its resemblance to the PEG model. 


Additional key themes and salient points related to collaboration that arose during interviews with LEA 
staff, ELP leadership, and center directors are summarized below. 


4.5.2 Sharing Best Practices 


Many LEA staff, ELP leaders, and center directors described the value of sharing information with and 
learning from others in their community. The most frequently identified focus of collaboration was 
teacher professional development supports, particularly coaching. Some ELP leaders and center directors 
also reported sharing strategies for engaging families and providing comprehensive services during 
regular meetings. 
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Some PEG center directors were positive about the collaboration between the LEA and ELPs with 
implementing ongoing coaching for teachers. One center director reported: 


I think they’ve [LEA] done a really great job. There have been plenty of opportunities for 
administrators to get some support. I’ve tried to really connect with the coach, and really want to 
know what they’re doing inside the classroom so I can support the teachers after the coach leaves. I 
want to understand what they’re thinking, and the feedback they’re giving them, because teachers can 
interpret things differently. I think there’s been a lot of support. 


4.5.3 Alignment of Regulations and Policies 


Many ELP leaders and LEA staff discussed the need to learn about the regulations and policies that 
governed work within each of their respective systems. LEAs, for example, were required to learn about 
licensing regulations and EEC policies related to ELP operations, while ELPs had to become familiar 
with LEA policies and practices. One ELP leader noted: 


Sometimes it’s confusing when I have different rules and regulations with EEC than with the public 
schools. They contradict each other sometimes. It’s hard to figure out what you’re supposed to do. It 
depends who is looking at you. You want to make sure you’re in compliance with everyone. 


Several ELP leaders also reported concerns that their LEA was focused too narrowly on curriculum, 
perhaps at the expense of other aspects of program quality. One community’s LEA staff also voiced a 
general desire to establish a more shared vision about what quality looks like and how it is defined and 
supported. Several ELP leaders and LEA staff noted that conversations about misaligned policies or 
practices—although difficult—tresulted in collaborative decision-making and more of a shared vision in 
their communities. 


4.5.4 LEA Role 


ELP leaders in one community commented on structural barriers to forming collaborative partnerships 
between the LEA and ELPs related to the role of the LEA in the partnership: 


I don’t think [the LEA’s monitoring role] works,” one ELP leader said, “especially to be true 
partnerships. Having the LEA be the auditor and program overseer doesn’t foster a partnership 
mentality; the LEA becomes the compliance organization, which makes the partnership go away.” 


ELP leaders in one community reported a competitive relationship between ELPs and LEAs to serve a 
finite/limited number of children. They believed the LEA preschool classrooms were taking children 
away from community organizations, related at least partially to the fact that children who attend LEA 
preschool programs were ensured slots in more desirable elementary schools in kindergarten. 


4.5.5 Communication among PEG Leaders 


While many center directors reported generally positive perceptions of communication among PEG 
leadership in the community, some also raised some concerns. For example, one center director noted: 


At times, it feels like a big chain of trickling information down. I feel like we don’t always get 
answers right when we would like them. It is a lot of people involved and a lot of different things on 
the table. 
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4.5.6 Communication within ELPs 


Several ELP leaders across communities reported challenges in communication and collaboration within 
their own programs, and suggested this is an area in which LEAs might provide professional development 
and support. At least two ELPs also reported a lack of technology to facilitate their own communications 
and information sharing processes; for example, some ELPs did not have work email addresses/systems 
for teachers, making communications difficult. 


4.5.7 Support for Center Directors 


Some communities held monthly meetings where center directors came together for mutual support and 
information sharing. In these communities, many center directors reported that these meetings were 
supportive and helpful, although some reported that they found it difficult to leave their programs to 
attend these meetings due to staffing shortages in the program. 


In one community, a center director reported that a site visit from the district PEG coordinator was very 
helpful, and something she would like to see continue. The director noted: 


I think what was good for me in the beginning was when [the LEA PEG coordinator] would come 
and do the site visits. That happened often in the beginning, but as we got more busy, it became more 
about going to larger meetings. Sometimes it was more helpful for me to have that one-on-one to 
clarify things and make connections. 


In communities that did not have regular center director meetings, some directors voiced the desire to 
have such meetings. One director suggested: 


I think it would be helpful to have PEG specific leadership meetings. Any of my leadership training 
has been on my own time. It would be helpful to know the struggles that other people are dealing with 
and dealing with this collaboration between so many groups. 


Several center directors voiced their satisfaction with the professional development they received as 
administrators during 2015-16, and those who had not received professional development indicated their 
desire to have these opportunities in year two. One center director commented: 


I know what I'd like to see in the future. I’d like more of an emphasis on helping center 
administrators. For me personally, the kind of professional development I want is a more clear path 
for my own education. I think there are few directors that don’t have master’s degrees. I think we 
could make it easier to obtain one. I don’t really feel like we’re reaching center directors at an 
administrator level. 


4.6 Wilder Collaboration Factors Inventory Findings 


The Wilder Collaboration Factors Inventory is a standardized tool to assess the elements of effective 
collaboration, as perceived by its members. The Wilder asks respondents to rate their collaboration on 40 
survey items that represent 20 factors that can contribute to the success of collaboration. (These factors 
were developed by the authors through a systematic review of empirical studies of collaboration.) On 
each survey item, respondents indicated their level of agreement using a five-point scale: (1) strongly 
disagree, (2) disagree, (3) neutral/no opinion, (4) agree, or (5) strongly agree. To score the Wilder, the 
responses on the survey items within each factor are averaged to form a factor score. The 20 factors are 
grouped under six “themes” or categories: environment, membership characteristics, process and 
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structure, communication, purpose, and resources. Exhibit 4.1 shows the six themes, which of the 20 
factors fall under each of the six themes, and in parentheses after each factor label, the number of survey 
items contributing to that factor. The authors of the Wilder suggest that factor scores can be loosely 
interpreted as follows: scores of 4.0 or higher show strength and probably do not need special attention; 
scores between 3.0 and 3.9 are borderline and may require attention; and scores of 2.9 or lower indicate 
concern and should be addressed. 


Exhibit 4.1: Wilder Collaboration Factors Inventory: Factors and Themes 


Theme Collaboration Factors 


Environment There is a history of collaboration/cooperation in the community (2) 
Collaboration is seen as a legitimate leader in the community (2) 
Political and social climate is favorable for collaboration (2) 
Membership There is mutual respect, understanding, trust among members of the collaboration (2) 
characteristics There is an appropriate cross-section of members in the collaboration (2) 
Collaboration members see collaboration as in their self-interest (1) 
Collaboration members are able to compromise (1) 
Collaboration members share a stake in process and outcome (3) 
Collaboration has multiple layers of participation from member organizations (2) 
Collaboration members demonstrate flexibility in considering options/approaches (2) 
Collaboration has developed clear roles and policy guidelines (2) 
Collaboration demonstrates adaptability in the fact of changing conditions(2) 
Collaboration demonstrates an appropriate pace of development (2) 
Communication Collaboration members demonstrate open and frequent communication (3) 
Collaboration members have established informal relationships/ communication links (2) 
Collaboration has concrete, attainable goals and objectives (3) 
Collaboration members have a shared vision of the goals of the work (2) 
The collaboration has a unique purpose in the community (2) 
Collaboration has sufficient funds, staff, materials and time to accomplish its goals (2) 
| Collaboration leaders are skilled at working with other people/organizations (1) 
Notes: Numbers in parentheses indicate the number of survey items within the factor. 


Resources 


4.6.1 Findings from the Wilder Collaboration Factors Inventory 


The findings in this section are based on the average scores (or ratings) for the PEG leaders for all of the 
survey items under each of the six broad themes. Please note that the term “organizational” or 
“organization” in this section refers to the local PEG collaborative versus an individual organization. In 
the results, each community has been assigned a letter to keep its identity confidential. Exhibit 4.2 shows 
the overall mean ratings for each of the themes and factors within the themes, for PEG overall and 
separately by community. 


Exhibit 4.3 compares the average ratings for each of the six themes overall and by community. Overall, 
the average ratings in Exhibits 4.2 and 4.3 suggest that the PEG communities have all developed strong 
collaborations, but that there is room for growth in some areas in some communities. 


Abt Associates Year 1 Massachusetts Preschool Expansion Grant (PEG) Evaluation Report J pg. 27 


COLLABORATION AND LEADERSHIP 


Exhibit 4.2: Average Wilder Ratings by Theme and Factor, Overall PEG and by Community 


Theme Community Total A B Cc D E 

There is a history of collaboration/cooperation in the community (2) 3.74 3.75 4.20 3.32 3.93 3.83 

EAVROR Teal Collaboration is seen as a legitimate leader in the community (2) 3.70 3.89 3.90 3.59 3.21 3.80 

Political and social climate is favorable for collaboration (2) 4.03 4.19 3.20 4.05 3.93 4.25 

Total for Environment 3.87 4.03 3.47 3.97 3.48 4.0 

There is mutual respect, understanding, and trust among members of collaboration (2) 3.56 3.64 3.40 3.36 3.43 3.79 

There is an appropriate cross-section of members in the collaboration (2) 3.46 3.36 3.90 3.05 3.43 3.79 

Membership Collaboration members see collaboration as in their self-interest (1) 4.37 4.50 4.60 4.45 4.14 4.17 
characteristics 

Collaboration members are able to compromise (1) 3.51 3.79 3.40 3.36 3.14 3.58 

Total for Membership characteristics 3.65 3.71 3.77 3.44 3.50 3.82 

Collaboration members share a stake in process and outcomes (3) 4.08 4.17 4.20 3.61 4.10 4.36 

Collaboration has multiple layers of participation from member organizations (2) 3.30 3.36 3.00 2.95 3.43 3.58 

Collaboration members demonstrate flexibility in considering options/approaches (2) 3.41 3.68 3.80 2.17 3.48 3.50 

Collaboration has developed clear roles and policy guidelines (2) 3.59 3.71 3.30 3.36 3.57 3.79 

Collaboration demonstrates adaptability in the face of changing conditions (2) 3.58 3.46 4.10 3.41 3.93 3.46 

Collaboration demonstrates an appropriate pace of development (2) 3.52 3.75 3.20 3.55 3.36 3.46 

Total for Process and structure 3.62 3.72 3.65 3.30 3.68 3.74 

Collaboration members demonstrate open and frequent communication (3) 3.61 3.95 3.60 3.24 3.76 3.47 

Communication Collaboration members have established informal relationships/communication links (2) 4.07 4.15 4.30 3.91 4.21 3.96 

Total for Communication 3.80 4.03 3.88 3.51 3.94 3.67 

Collaboration has concrete, attainable goals and objectives (3) 4.04 4.05 4.33 3.61 4.29 4.17 

Collaboration members have a shared vision of the goals of the work (2) 3.79 3.82 3.90 3.45 4.14 3.79 

The collaboration has a unique purpose in the community (2) 3.88 3.68 4.00 3.68 3.86 4.27 

Total for Purpose 3.92 3.88 4.11 3.58 4.12 4.09 

Collaboration has sufficient funds, staff, materials and time to accomplish its goals (2) 3.31 3.39 2.90 3.36 3.43 3.25 

Resources Collaboration leaders are skilled at working with other people/organizations (1) 3.78 3.64 4.20 3.18 4.29 4.00 

- Total for Resources 3.46 3.48 3.33 3.30 3.71 3.50 


NOTES: Factor scores of 4.0 or higher show strength and probably do not need special attention; scores between 3.0 and 3.9 are borderline and may require attention; and scores of 2.9 or lower indicate concern and 
should be addressed. Numbers in parentheses indicate the number of survey items within each factor. 

SAMPLE: The N ranged from 5 to 14 respondents by community; for overall PEG, N=29 respondents. 

SOURCE: Wilder Collaboration Factors Inventory of PEG ELP and LEA Directors (Spring 2016). 
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Exhibit 4.3: Average Wilder Rating by Theme, Overall PEG and by Community 
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READS AS: On average, ELPs in PEG rated their communities’ collaborative environments as 3.9 out of 5. 

NOTES: Authors of the Wilder suggest that factor scores of 4.0 or higher show strength and probably do not need special attention; scores 
between 3.0 and 3.9 are borderline and may require attention; and scores of 2.9 or lower indicate concern and should be addressed. 
SAMPLE: The N ranged from 5 to 14 respondents by community; for overall PEG, N=29 respondents. 

SOURCE: Wilder Collaboration Factors Inventory of PEG ELP and LEA Directors (Spring 2016). 


Theme: Environment 


PEG leadership teams were quite positive about the environment for their collaboration, with a mean 
rating of 3.87 out of 5.00 across all factors and communities. Leaders reported a broadly favorable 
political and social climate for their work (4.03), although there was less widespread agreement that there 
was a history of working together to solve common problems prior to PEG collaboration (3.74) or that 
member organizations were seen as legitimate leaders on the issues they worked to address (3.70). 


Perspectives varied somewhat by community, although no clear pattern emerged. PEG leadership in one 
community (D) was less likely to agree that there was a broadly favorable social climate for their work 
than the other four. Leadership in another community was less likely to agree that their community had a 
history of collaboration (community C), and leaders in yet another community had lower agreement that 
members of the collaborative were seen as local leaders (community B). 


Theme: Membership Characteristics 


PEG leadership teams were generally positive about their collaborations’ membership characteristics, 
with a mean rating of 3.65 across all factors and communities. Leadership agreed that participating 
organizations saw collaboration as being in their best interest (the mean of 4.37 on this factor was the 
highest rated in the inventory), but were less in agreement that their collaborative had an appropriate 
cross-section of members (3.46). 


There was some variation by community, particular on the subject of the collaboration including an 
appropriate cross-section of members. Of note, community C’s mean was lower (3.05) than the overall 
mean of 3.46. 
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Theme: Process and Structure 


PEG leadership teams were generally positive about their collaborations’ process and structure, with a 
mean score of 3.62 across all factors and communities. PEG leadership was neutral about whether their 
collaborative provided for multiple layers of participation (3.30), neutral about whether their collaborative 
took on work at an appropriate pace (3.52), and more positive but still fairly neutral about if the 
collaborative had developed clear roles and guidelines (3.59). Two communities were the most negative 
on each of the indicators (C and D), and one community was consistently the most positive (A). 


Theme: Communication 


Overall, PEG leadership rated the communication among the members of the collaborations quite 
positively, with a mean rating of 3.80 across all factors and communities. There was some variation by 
community; in three communities, ratings were near 4.00 across all factors. PEG leaders were generally 
more likely to report success in establishing informal relationships and communication links (4.07) than 
success in demonstrating open and frequent communication (3.61). 


Theme: Purpose 


PEG leadership teams were generally positive about their collaborations’ sense of purpose, with a mean 
rating of 3.92 across all factors and communities. PEG leadership varied by community in their 
perceptions of the goals, mission, and shared vision of their local collaborative. PEG leadership in one 
community (C), for example, were somewhat less likely than others to agree that their collaborative had 
concrete, attainable goals and objectives (mean rating of 3.61 versus overall mean of 4.04), that it had a 
unique purpose in their communities (3.68 versus 3.88), or that members of the collaborative had a shared 
vision for its future (3.45 versus 3.79). 


PEG leadership in three communities (A, B, and D) was generally quite positive about the state of their 
shared mission, reporting strong agreement with all three relevant statements. Respondents in two of these 
communities unanimously agreed that their collaborative had concrete, attainable goals and objectives, 
and PEG leadership in one of the two communities was unanimously positive about their shared vision 
for the future. 


Theme: Resources 


PEG leadership teams were less positive about the state of their resources than they were about anything 
else, with a mean rating of 3.46 across all factors and communities. Leadership was generally less 
positive that their collaboration had sufficient resources at hand (3.31), although they reported general 
satisfaction with their local leadership (3.78). 


Perceptions varied by community. PEG leadership in one community was less positive about the skills of 
their leadership (C’s mean was 3.18 compared to the overall mean of 3.78). Leadership in another 
community was more positive than the rest (4.20) about the skills of their leadership, although this 
community was also the least positive about whether their collaborative had access to adequate resources 
(D’s mean was 2.90 compared to the overall mean of 3.27). 
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5. Teacher Supports 


During 2015-16, PEG LEAs and ELPs provided an array of supports to their teachers aimed to achieve 
multiple goals including: building teacher instructional competencies, increasing teachers’ sense of 
professionalism and job satisfaction, and improving teacher stability. Ultimately, the teacher supports are 
expected to lead to higher classroom quality, which in turn will lead to improved child learning and 
development. 


This chapter includes an overview of the characteristics of PEG teachers; a description of the supports 
that PEG teachers received during 2015—16 including: compensation; professional development, 
coaching, and planning time; and reports on two early teacher self-reported outcomes (self-efficacy and 
job satisfaction). 


5.1 Summary of Findings 


e PEG aimed to compensate its teachers in line with the level of pay for public school preschool 
teachers. PEG lead teachers were paid an average of $54,246 in 2015-16 (not including fringe 
benefits), which was in line with what preschool teachers were paid by school districts in the PEG 
communities and higher than typical teacher salaries in Head Start programs and other community- 
based programs in Massachusetts. 

e PEG LEAs and ELPs were expected to provide supports to teachers, including professional 
development and coaching. Programs were also encouraged to provide paid time for teachers to use 
for instructional planning. 

— Onaverage, PEG lead teachers were offered 23 hours of professional development training in the 
first year, across a variety of topics. The level of professional development supports varied across 
communities and ELPs. 

— Teachers were provided with coaching by LEA staff in four of the five PEG communities. The 
amount of coaching teachers received varied across communities and programs. About 44 percent 
of PEG lead teachers reported receiving more than 20 hours of coaching in the first year, while 
almost one-third of teachers (32 percent) reported receiving 10 or fewer hours of coaching. 

— Over 60 percent of PEG lead teachers reported weekly paid release time for instructional 
planning. 

e PG teachers reported moderate to high satisfaction with being a PEG teacher and high levels of 
commitment to teaching. They generally felt confident about their ability to work with students and 
communicate with parents. 

e In later years, the evaluation will examine whether the financial and professional supports for PEG 
teachers result in greater teacher stability and skill. 


5.2 Characteristics of PEG Teachers 


A core PEG program goal is to establish classrooms that provide children with a high-quality learning 
environment supported by a workforce of qualified and experienced educators. As required by the grant, 
all lead teachers surveyed had at least a bachelor’s degree and 23 percent also had a Master’s degree (23 
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percent) (see Exhibit 5.1).° Fewer PEG assistant teachers than lead teachers had at least a bachelor’s 
degree (16 percent) and almost one-third (31 percent) had no more than a high school degree or GED. 


Exhibit 5.1: Characteristics of PEG Teachers, 2015-16 


Lead Teachers Asst. Teachers 
n % n % 
Educate 
High school diploma or GED 0 0% 10 31% 
Associate’s/Technical/Vocational Degree 0 0 16 50 
Bachelor's degree 30 i 5 16 
Master's degree 9 23 1 3 
1 year 3 8% 7 21% 
2-5 years 12 31 14 42 
6-10 years 12 31 5 15 
More than 10 years 12 31 u 21 
English 39 100% 32 97% 
Spanish 10 26 15 45 
Creole 2 5 0 0 
French 1 3 0 0 
Haitian 1 3 0 0 
Other 3 8 1 3 
ther eee 

Externally Employed (Part- or Full-Time) 8 21% 5 16% 
Currently in School as a Student 8 21% 13 41% 


READS AS: Zero lead teachers and 10 assistant teachers (or 31% of assistant teachers) held a high school diploma or a GED as their highest 
level of education. 

NOTES: Percentages were rounded to the nearest whole number; Responses for “Language Spoken” do not sum to 100 percent because 
multiple responses were permitted. 

SAMPLE: For lead teachers, N=38-39, missing=0-1. For assistant teachers, N=32-33, missing=0-1. For all teachers, N=71-72, missing=0-1. 
SOURCE: PEG Teacher Survey (Spring 2016). 


All PEG lead teachers had at least one year of experience as a preschool teacher, and 62 percent had at 
least six years of experience. PEG assistant teachers were less experienced than lead teachers; the 
majority of assistant teachers (63 percent) were in their first five years of teaching. 


The level of education of PEG lead teachers was higher than what is reported nationally for programs 
serving disadvantaged preschool children; nationally 45 percent of center-based teachers and caregivers 
serving children ages 3-5 had a bachelor’s degree or higher (ACF OPRE, 2013). It is also higher than the 
30 percent that was recently reported for center-based teachers in Massachusetts; if the 30 percent reflects 


> Teacher characteristics were reported on the teacher survey, which was completed by 39 lead teachers 


representing 35 of the 48 PEG classrooms and 33 assistant teachers from 27 classrooms. 
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both assistant and lead teachers, the comparable PEG proportion is 63 percent, which is still double the 
state rate (Marshall, Dennehy, Johnson-Staub, & Wagner-Robeson, 2005; Massachusetts, 2009). 


About one-quarter of PEG lead teachers (26 percent) and about two-fifths of PEG assistant teachers 

(45 percent) reported speaking Spanish, though not necessarily in the classroom. As already described, 
about one-third of PEG children came from a home where the primary language was not English, with the 
predominant non-English home language being Spanish. 


Early childhood teachers may take additional jobs to supplement their incomes and/or enroll in school to 
further their education. Approximately one-fifth of PEG lead and assistant teachers reported holding 
either part- or full-time external employment (on top of their PEG roles). Twenty-one percent of lead 
teachers and 41 percent of assistant teachers reported that they were also currently in school. 


5.4 Compensation 


The average total teacher compensation for a PEG classroom lead teacher was $67,475 in 2015-16, a 
combination of $54,246 in salary and $13,229 in fringe and other benefits. Across the five PEG 
communities, the average lead teacher salary (excluding fringe benefits) ranged from $54,000 to $71,000. 
Some programs used funds other than PEG to pay portions of their teacher salaries. A federal report 
released in 2016 found that on average Massachusetts elementary school teachers were paid $71,240, 
preschool special education teachers were paid $55,860, Head Start teachers were paid $28,078, and other 
child care teachers were paid $24,980 (US Department of Education, 2016). 


Despite PEG’s efforts to provide higher teacher compensation, teachers had a range of perceptions of the 
adequacy of their salaries. Most lead teachers surveyed (64 percent) were somewhat or very satisfied with 
their salaries while 13 percent of lead teachers reported that their salaries were inadequate. Total 
compensation for PEG assistant teachers averaged $43,248. Assistant teachers had lower levels of 
satisfaction compared to lead teachers with their salaries. About half of assistant teachers (45 percent) 
were somewhat or very satisfied with their salaries, and 21 percent of assistant teachers reported that their 
salaries were inadequate. 


5.5 Professional Development 


In early education, there is a common assumption that in-service teacher training affects teacher practices 
in the classroom, and those practices in turn result in positive benefits for children. There are a limited set 
of findings that show that teacher training is related to improved quality of early childhood programs 
(Burchinal, Howes, & Kontos, 2002; Burchinal, Cryer, Clifford, & Howes, 2002) and more sensitive 
interactions with children (Clarke-Stewart, Vandell, Burchinal, O’Brien, & McCartney, 2002). While 
there is some research about what types of training strategies are most likely to be effective, there is some 
evidence suggesting that more intensive and longer duration training is likely to be better than brief 
training (Tout, Zaslow, & Berry, 2006; Loeb, Rouse, & Shorris, 2007). 
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5.5.1. Hours of Professional Development 


A primary support provided to teachers in the first year of PEG was professional development.’ PEG 
LEAs and programs were allowed to design their own professional development programs, which could 
vary in intensity and topics covered. At the same time, LEAs and ELPs were encouraged to collaborate 
within their communities on professional development approaches and activities. Most center directors 
(75 percent) reported that professional development to teachers was provided primarily by school district 
staff. Less often, ELP and center staff delivered the professional development. 


On average, PEG lead teachers reported attending 23 hours of professional development training during 
2015-16. However, there was substantial variation among teachers in the amount of training, ranging 
from a few to 50 hours. There also were substantial differences across PEG communities in the amount of 
professional development that teachers reported receiving, with average hours ranging from 10 to 29 
(Exhibit 5.2). 


Exhibit 5.2: Average Hours of Professional Development Received by PEG Lead Teachers, Overall 
PEG and by Community 


60 
@ AIPEG 


50 


PEG Communities 


Hours of Professional Development 


Overall PEG A B C D E 
Community 


READS AS: PEG lead teachers reported receiving between 4 and 50 hours of professional development over the past year, with an average of 
23 hours. 

SAMPLE: N ranged from 3 to 10, by community. For overall PEG, N=25, missing=13. 

SOURCE: PEG Teacher Survey (Spring 2016). 


Assistant teachers reported receiving, on average, 61 percent of the hours of professional development 
received by lead teachers (Exhibit 5.3). The average amount of professional development for assistant 


* In PEG, professional development is an encompassing term used to refer to in-service training and education to 


increase teaching skills, knowledge of a particular subject area, or to learn to implement a particular curriculum. 
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teachers was 14 hours, although the averages varied across communities. In the same community where 
lead teachers reported the most professional development, assistant teachers also reported receiving the 

most training—an average of 26 hours. In the other two communities for which data were reported, the 

average amount of professional development received by assistant teachers ranged between 6 and 

14 hours. 


Exhibit 5.3: Average Hours of Professional Development Received by PEG Assistant Teachers, 
Overall PEG and by Community 
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READS AS: Overall, PEG assistant teachers reported receiving between 1 and 40 hours of professional development, with a mean of 
14 hours. 
SAMPLE: N ranged from 2 to 4 by community for Communities A, D and E. For overall PEG, N=12, missing=19. Only one assistant teacher in 
each of Communities B and C reported their hours of professional development, so their data is not reported. 
SOURCE: PEG Teacher Survey (Spring 2016). 


5.5.2 Training Topics 


A substantial majority of PEG lead teachers (over 80 percent) reported receiving training on four topics in 
2015-16: training on guiding and interacting with young children, training on implementing specific 
curricula, training on general content instruction, and training on conducting formative child assessments 
(Exhibit 5.4). Fewer teachers (less than 50 percent) reported receiving training on state standards and 
special education referrals. 
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Exhibit 5.4: Topics Covered in PEG Professional Development and Usefulness as Reported by 
Lead Teachers 


Percentage of Lead Teachers Who Reported that: 


Topic Was Addressed Professional 
in Professional Development Was Very 

Professional Development Topics Development Effective 
Guiding and interacting with young children 92% 56% 
Curriculum-specific training 84% 64% 
General content instruction (language/literacy/math instruction) 84% 60% 
Conducting formative child assessments through observation, child 80% 36% 
screening, and/or assessments 
Supporting children’s social/emotional development 16% 56% 
Understanding child development 16% 52% 
Classroom organization and learning environments 12% 36% 
Health, safety, and nutrition 12% 32% 
Training in using particular assessment tools (i.e., ASQ, TS Gold, 12% 36% 
DIBELS) 
Behavior Management 68% 32% 
Communicating with parents 68% 40% 
Working with diverse populations 68% 44% 
Supporting children with special needs in the classroom 60% 20% 
Training in using particular tools to understand the classroom 56% 24% 
environment (ECERS, CLASS, etc.) 
Training on state standards 48% 20% 
Special education referrals 36% 8% 


READS AS: Ninety-two percent of lead teachers reported that PEG professional development addressed guiding and interacting with young 
children, and 56 percent of lead teachers reported that the professional development coverage of the topic was very effective. 

SAMPLE: N=25, missing=13. 

SOURCE: PEG Teacher Survey (Spring 2016). 


5.5.3 Perceived Effectiveness of Professional Development 


In general, PEG lead teachers reported that some of the professional development that they received was 
useful in 2015-16. There were five areas in which the professional development was judged to be 
effective (by 50 percent or more of teachers), including training on guiding and interacting with young 
children, specific curricula, content instruction, supporting children’s social/emotional development, and 
understanding child development more generally (Exhibit 5.4). In focus groups, lead teachers reported 
that, regardless of the topic, they especially appreciated training that provided concrete strategies and 
resources for classroom management and instruction. 


The professional development related to supporting special education referrals was perceived to be 
ineffective by nearly all lead teachers, and less than one-quarter of teachers reported that the professional 
development around supporting children with special needs, state standards, and using specific child 
observation tools to be useful. 
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5.5.4 Challenges in Implementing Professional Development 


Center directors noted that coordinating teacher training with the public schools introduced some 
challenges for implementing professional development in 2015-16. The challenges included scheduling 
training during times when the schedules of the PEG programs and the public schools differed and some 
reports of perceived lack of buy-in to the PEG program among public school staff. Center directors noted 
other administrative challenges in planning and providing professional development, such as finding 
sufficient time for professional development in light of restrictions on the number of days that programs 
could close for teacher training and insufficient budgets. Finally, center directors noted that the 
professional development provided by school district staff was not always customized to the needs of 
teachers and that the training sometimes focused too narrowly on topics related to meeting state Quality 
Rating and Improvement System (QRIS) requirements. 


Lead teachers in most PEG communities also raised concern that assistant teachers received less 
professional development than teachers did. They recommended that the same level of professional 
development be offered to all staff in future school years. 


5.6 Coaching 


In the PEG model, coaching is one important means by which support is delivered to teachers to improve 
knowledge and practice. Although coaching was not required in year one, it was strongly encouraged and 
added as an expectation in year two. This expectation is aligned with Level 4 of the Massachusetts QRIS 
which requires programs to “demonstrate systematic opportunities for teachers to engage in reflective 
teaching practices through the use of peer groups, coaches and/or mentors.” 


In recent years, there is growing recognition and evidence of the potential value of coaching for 
supporting improvements in teaching in early childhood classrooms. Though there is little information 
about the specific components of coaching, the broader body of literature on professional development 
efforts that include coaching is extensive. In 2011, Isner et al. conducted a literature review to examine 
48 studies on the effects of professional development with a coaching element. Of the studies that 
examined staff outcomes, about 60 percent reported positive findings on one of the following: knowledge, 
attitudes, and satisfaction with the coaching. Most of the studies also included a measure of observed 
quality and instructional practices with children and, of those, 87 percent reported positive outcomes. 
Finally, about half of the studies included child outcomes. Of those, just over half found positive language 
and literacy outcomes for children, 28 percent found positive outcomes on behavioral measures, and just 
one study reported a positive outcome on math. Isner et al. (2011) concluded that the existing research 
suggests positive associations between coaching and quality improvements in early education settings. 


PEG communities were allowed to design their own coaching model in terms of content, dosage, and 
strategies. Coaches met with individual PEG teachers or teaching teams to offer instructional support and 
other classroom management support to improve instruction. While PEG coaches did not exist in one 
community during 2015-16, at least one of the center directors in that community offered instructional 
support to the PEG teachers in her building. 


5.6.1 PEG Coaches 


In the four communities with PEG coaches, the coaches were hired by the school district. All of the 
coaches had expertise in early childhood education and development. With one exception, coaches were 
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former early childhood teachers. In three communities, PEG coaches were hired from an existing pool of 
district coaches or experienced district teachers. 


PEG coaches received professional development related to coaching, instructional leadership, or teaching 
in prekindergarten classrooms. Coaches in three communities attended team meetings in which a 
supervisor provided professional development or other kinds of support directly related to the 
responsibilities of coaches. In two communities, coaches were provided with mentorship opportunities in 
which the mentor observed the coach in action and debriefed with the coach about his/her work with the 
teacher. In one case, the mentor modeled the process of providing feedback to teachers. 


Across the four communities, coaches had other responsibilities in addition to coaching. Coaches in two 
communities were responsible for developing curriculum for the PEG and district preschool classrooms. 
All coaches reported monthly meetings with district supervisors and monthly meetings with PEG 
leadership. Coaches differed in their reports of the relative amount of time spent in meetings, but stated 
that there was at least one meeting per week. These meetings included those with district supervisors and 
district curriculum and instruction teams. Importantly, in three communities, coaches met with other PEG 
or district coaches; these coaches reported during interviews that these meetings contained essential 
support and training content. 


5.6.2 Coaching Activities 


Coaching Strategies 

PEG coaches reported using some common strategies across the five communities. All coaches reported 
using a sequence of “observe, reflect, and discuss” with teachers. First, coaches observed the classroom 
and the teachers’ practices within the classroom as part of coaching, reflected on their observations, and 
then met with the teacher to discuss what the coach observed. While observing, coaches gathered 
information either in the form of written notes, photographs, or videos. Afterward, coaches reflected upon 
what s/he observed and created a set of key points to discuss with the teacher or teaching team. Finally, 
the coach met with the teacher or teaching team to discuss the observation notes and other information. In 
this meeting, coaches and teachers often generated a set of goals and next steps. 


Coaching Topics 

Lead teachers reported that the PEG coaching addressed a range of topics. The most common topic of 
coaching (reported by 79 percent of teachers) was about general content instruction (Exhibit 5.5). Most 
teachers also received coaching on how to use specific curricula, the general classroom environment, and 
on supporting children’s social emotional development. Only 21 percent of teachers received coaching on 
special education referrals. Coaches reported that earlier in the school year, the coaching was more likely 
to focus on classroom organization and behavior management. Later in the year, coaching was more 
likely to focus on curriculum content and pedagogy. 
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Exhibit 5.5: Topics of Coaching Reported by PEG Lead Teachers 


Lead Teachers 


N % 
General content instruction (language and literacy/math instruction) 27 19% 
Curriculum-specific training 26 16% 
Classroom organization and learning environment 26 16% 
Supporting children’s social/ emotional development 24 1% 
Behavior Management 22 65% 
Follow-up to professional development trainings 16 47% 
Supporting children with special needs in the classroom 12 35% 
Supporting English Language Learners 11 32% 
Using information from child formative assessment, screeners and assessments 10 29% 
Aligning classroom practices with state standards 10 29% 
Working with diverse populations 10 29% 
Using information from formal classroom observation tools (i.e., CLASS) 9 26% 
Special education referrals 7 21% 


READS AS: Twenty-seven lead teachers, or 79% of PEG lead teachers, received coaching on general content instruction. 
SAMPLE: N=34, missing=2. 
SOURCE: PEG Teacher Survey (Spring 2016). 


About half (47 percent) of surveyed lead teachers reported that at least one coaching session was directly 
related to professional development trainings that they received. According to interviews with coaches, in 
three communities teachers were intentionally involved in determining what coaches would observe and 
address in coaching sessions. Teachers in two communities completed a formal needs assessment at the 
beginning of the school year for coaches to review. In another community, the PEG coaches set a loose 
structure for the 2015—16 coaching sessions—a list of topics that all coaches were to cover with teachers 
over the course of the year; other topics and teacher needs could be addressed as well. 


5.6.3 Coaching Dosage 


Coaches typically observed teachers in their classrooms and then met with them to discuss the 
observation. In two of the communities where coaching was provided, most lead teachers reported being 
observed and meeting with their coach monthly. In the other two communities, lead teachers reported 
being observed and meeting with their coach a few times per year. Typically, coaches met with teachers 
one-on-one, but 23 percent of teachers reported that they sometimes met with the coach with a group of 
teachers. 


In terms of the total amount of coaching teachers reported receiving, the dosage varied widely: 14 percent 
of teachers reported receiving more than 40 hours of coaching, while 32 percent of teachers received 10 
or fewer hours (Exhibit 5.6). 
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Exhibit 5.6: Hours of Coaching Reported by PEG Lead Teachers, Overall PEG and by Community 
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READS AS: Thirty-two percent of PEG teachers reported receiving between 0 and 10 hours of coaching over the last year. 
SAMPLE: N ranged from 16 to 3, by community. For overall PEG, N=37, missing=1. 
SOURCE: PEG Teacher Survey (Spring 2016). 


5.6.4 Supervision and Documentation of Coaching 


Teacher progress resulting from coaching was formally tracked in one of the four communities that 
provided coaching. In this community, coaches kept logs that helped them identify what to follow-up on 
in subsequent sessions and also allowed them to note any growth/progress that was observed. An 
observation tool was used in one coaching session that focused specifically on how teachers were 
implementing the curriculum. Information gathered during this session was used to create action plans for 
individual teachers. In the other three communities, tracking was less systematic. Coaches followed up 
with teachers informally, but did not record whether progress was made on any goals or areas of 
improvement that were identified in previous coaching sessions. 


5.6.5 Effectiveness of Coaching 


Teacher perceptions of the utility of coaching differed across communities. In focus groups, teachers in 
one community reported that the coaching was highly beneficial. Teachers in two other communities 
reported that the coaching was useful, but not well-aligned with what was happening in the classroom; 
these teachers wanted coaches to provide more actionable feedback targeted to specific teacher practices 
or child learning outcomes. 
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5.6.6 Challenges Implementing Coaching 


Coaches reported several barriers to the delivery of coaching. 


e Insufficient time. Coaches and teachers in two communities had difficulty finding enough sufficient 
staff classroom coverage for necessary coaching activities and communications. Coaches and teachers 
reported that this barrier arose due to incomplete or shifting staffing in PEG classrooms. 

e Space. The physical space presented an additional barrier to coaching. In two communities, some 
teachers were geographically isolated, with individual classrooms scattered across a range of early 
childhood education settings. This barrier increased travel time for the coaches and prevented 
teachers from forming relationships with each other that could reinforce the support they were 
receiving from the coaches. In one community there was not enough space available in the center for 
private follow-up meetings between the coach and teachers. In some cases, coaches had to wait one to 
two days after observations to have this type of meeting. 

e Understanding regulations and context. Coaches, teachers, and ELP leadership in two communities 
reported that coaches were sometimes unfamiliar with ELP regulations and context. The coach’s 
unfamiliarity with regulations decreased the utility of the strategies and support that he/she was able 
to provide. Given that LEAs and ELPs are regulated by different state agencies, there were some 
instances where coaches’ suggestions were not only poorly-suited to the ELP context but contradicted 
state licensing requirements for the ELP center or agency. For example, one ELP leader described a 
scenario in which the coach suggested that a teacher leave the room in order to plan, but if the teacher 
had done this, the adult-child ratio required by state licensing standards would not have been met. 


5.6.7 Coach-Recommended Improvements 


According to coaches, relationship building was important for coaching success. They reported that one- 
on-one meetings between teachers and coaches, as well as meetings with entire teacher teams, facilitated 
high quality coaching. Coaches reported that they would like to meet more with other PEG coaches in 
future school years, work with a literacy instruction specialist, and receive training related to the CLASS 
measure. Additionally, coaches in two communities would like to meet with PEG coaches in other 
communities to share best practices and learn more about how they organize their work. In two other 
communities, coaches desired more professional development from a literacy instruction specialist. In one 
community, coaches discussed the intention to create a contract in 2016-17 that outlined the 
responsibilities of coaches and teachers. 


5.7 _ Paid Instructional Planning Time 


The PEG model did not include a requirement that teachers have paid time for planning, but EEC 
encouraged PEG programs to provide this support as a strategy for developing high-quality programming 
for children. About half of the lead teachers (53 percent) reported receiving 1-5 hours per week of formal 
release time for planning, while a smaller proportion (8 percent) received 6 or more hours of planning 
time.” Approximately one-third of lead teachers (39 percent) received no formal release time per week 
(Exhibit 5.7). Assistant teachers reported similar hours of paid planning time. 


° Teachers may have interpreted the definition of formal planning/release time differently; it is possible that some 


teachers counted time spent planning in the classroom while children were present and some did not. The 
survey described this as “time from your program for planning/other non-instructional work responsibilities.” 
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Exhibit 5.7: Weekly Hours of Paid Planning Time Reported by PEG Lead Teachers, Overall PEG 
and by Community 
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NOTE: Since community B only includes responses from 3 lead teachers, the 67% in the exhibit reflects the responses of 2 teachers. 
READS AS: Thirty-nine percent of PEG lead teachers reported receiving no weekly planning time, and 53 percent of PEG lead teachers 
reported receiving one to five hours of planning time per week. 

SAMPLE: N ranges from 3 to 15, by community. For overall PEG, N=38, missing=1. 

SOURCE: PEG Teacher Survey (Spring 2016). 


Among lead teachers who received some formal release time, the most commonly-reported time during 
the day during which they had release time was during the children’s nap (85 percent), followed by other 
break periods such as lunch. 


5.7.1. Teacher Perceptions of Planning Time 


The majority of lead teachers (87 percent) and assistant teachers (76 percent) reported that paid planning 
time was “very important” to helping do their jobs. However, half of lead teachers (50 percent) felt they 
did not have enough paid planning time to complete their responsibilities. The majority of assistant 
teachers (76 percent) reported that they had just enough paid planning time. 


According to the center directors, the key barrier to paid planning time was coverage of classrooms so 
that the adult-child ratio was maintained. Coverage was affected by incomplete or shifting staffing in PEG 
classrooms and teacher absences within the PEG classroom or the center in which the PEG classroom was 
located. 


5.8 Early Teacher Outcomes 
5.8.1. Job Satisfaction and Self-Efficacy 


One of the goals of the PEG program is to support teachers’ sense of professionalism and self-efficacy 
through professional development, coaching, and compensation. Ultimately, it is expected that teachers 
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who feel more positively about their jobs and their own competencies will remain in the program, thereby 
providing the program continuity and stability. Some studies have found that teachers with high self- 
efficacy are more likely to believe that they can have an impact on what and how the children in their 
classrooms learn (Tschannen-Moran, Hoy, & Hoy, 1998). Importantly, teacher self-efficacy has also been 
found to have a positive association with teacher willingness to implement newly learned tasks and 
strategies, choice of activity, task persistence and effort expenditure (Allinder, 1994; Fuchs, Fuchs, & 
Bishop, 1992; Guskey, 1988; Pajares, 1997; Woolfolk, Rosoff, & Hoy, 1990). 


Since teacher retention will not be examined until the second year of the PEG program, the Year 1 
evaluation focused on teacher job satisfaction and self-efficacy toward the end of the first year of program 
implementation. Most PEG lead teachers were either very satisfied (49 percent) or somewhat satisfied 

(44 percent) with their jobs. A high percentage of teachers were satisfied with their decision to be a 
preschool teacher; 74 percent of lead teachers and 67 percent of assistant teachers reported that they 
would “certainly” become a teacher again if they were to go back to their college days and start over 
again. Also, most lead teachers (77 percent) and assistant teachers (79 percent) planned to remain in 
teaching “as long as they are able.” 


PEG teachers also expressed a high level of confidence in their ability to work with students (Exhibit 5.8). 
This confidence included feeling that they made a significant difference in students’ lives and that they 
were successful at reaching even the most difficult students. Teachers were less confident about their 
influence on their students’ achievement and motivation, relative to the influence of their home 
environments and peers. However, teachers were highly confident in their ability to communicate with 
parents and guardians; on average, teachers rated their ability to communicate with parents about various 
topics as above 4.6 on a 5-point scale. 
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Exhibit 5.8: Lead Teacher Confidence Regarding Interactions with Students 


Disagree or Agree or 
Disagree Agree Agree 
very Disagree justa very 
strongly just a little little strongly 
(%) (%) (%) (%) 
Positive Statements about Teacher Abies 
| feel that | am making a significant educational difference in the 0% 0% 3% 97% 
lives of my students. 
If | try really hard, | can get through to even the most difficult and 0% 0% 10% 90% 
unmotivated students. 
| am successful with the students in my class. 3% 0% 5% 92% 
| usually know how to get through to students. 0% 0% 3% 97% 
There is a limited amount that | can do to raise the achievement 82% 5% 5% 8% 
level of students. 
Children are so private and complex, | never know if | am getting 67% 15% 10% 8% 
through to them. 
| feel as though some of my students are not making any 64% 13% 13% 10% 
academic progress. 
Most of a student's motivation depends on the home environment, 61% 18% 8% 13% 
so | have limited influence. 
My students’ peers influence their academic performance more 59% 32% 8% 0% 
than | do. 
| am uncertain how to teach some of my students. 59% 3% 26% 13% 
My students’ peers influence their motivation more than | do. 55% 32% 11% 3% 


READS AS: Ninety-seven percent of PEG lead teachers agreed or agreed very strongly that they felt they were making a significant 
educational difference in the lives of their students, and 3 percent agreed just a little. 

SAMPLE: N=37-39. 

SOURCE: PEG Teacher Survey (Spring 2016) using items from the Teacher Self-Efficacy for Teaching Scale (Hoover-Dempsey, et al., 2002). 


5.8.2 Teacher Perceptions of Support from PEG Administrators 


More than half of lead teachers surveyed (60 percent) reported that ELP and center directors were “very 
supportive” in helping them manage their classrooms effectively, while another 23 percent of teachers 
characterized their leadership as “somewhat supportive.” Teachers were similarly positive in focus 
groups; most teachers reported that they generally felt supported by their ELP and center directors. 


Teachers were somewhat less positive when asked about several specific aspects of support from their 
program administration (Exhibit 5.9). Notably, 38 percent of teachers disagreed (somewhat or strongly) 
that they had the support they needed to teach students with special needs. This finding is consistent with 
teacher reports of having little training focused on special education referrals. 
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Exhibit 5.9: Lead Teacher Perceptions of Support and Leadership from PEG Program 
Administration 
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program he or she wants and student conduct and backs me__ recognized for a job well done. _ teach students with special 
has communicated it to the staff. up when | need it. needs. 


Strongly agree = Somewhat agree Somewhat disagree Strongly disagree 


READS AS: Forty-eight percent of lead teachers reported strongly agreeing with the statement “The director knows what kind of program he or 
she wants and has communicated it to the staff,” 40% of teachers agreed somewhat, and 13% of teachers disagreed somewhat. 

SAMPLE: N=38-39, missing=0-1. 

SOURCE: PEG Teacher Survey (Spring 2016). 


5.8.3 Additional Teacher Perceptions 


Overall, lead teachers reported high or moderate satisfaction with the curricula used in their classrooms 
(Exhibit 5.10). Between one-quarter and one-third of lead teachers were neutral or unsatisfied with 
several other aspects of the support available for instruction: support for individualization of instruction 
(34 percent), support for assessment of student progress (31 percent), and cultural appropriateness 

(26 percent). 
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Exhibit 5.10: Lead Teacher Satisfaction with Aspects of PEG Curricula 
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READS AS: Fifty-four percent of lead teachers reported being very satisfied with PEG Curricula’s effectiveness on supporting children’s 
development. 

SAMPLE: N=39, missing=0. 

SOURCE: PEG Teacher Survey (Spring 2016). 


Lead teachers also identified several challenges specific to their programs. A notable proportion of 
surveyed teachers reported student absenteeism and tardiness as serious problems (26 percent and 

15 percent, respectively). Some teachers also reported that lack of parental involvement and teacher 
absenteeism were serious problems (10 percent and 8 percent, respectively). Furthermore, 11 percent of 
lead teachers strongly agreed and 34 percent somewhat agreed that routine duties and paperwork 
interfered with their job and teaching. 
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Providing children high quality instructional programming within the classroom is a key goal of the PEG 
program. In the PEG model, two elements of instructional quality are the utilization of a developmentally 
appropriate, comprehensive curriculum, aligned to the state’s standards, to guide instruction and 
utilization of an assessment system to track student progress and guide instruction. 


This chapter presents information about the curricula and formative assessments used in PEG classrooms 
during 2015-16, as reported by teachers and center directors. This chapter also presents descriptive 
information about the activities and instructional groupings in the PEG classrooms based on independent 
observations of the classrooms by the study team. 


6.1 Summary of Findings 


e All PEG programs surveyed used developmentally-appropriate curricula. In three of the five PEG 
communities, all of the programs in each community used the same comprehensive curriculum. In the 
other two communities, different programs used different curricula. 

e All of the PEG programs conducted formative assessments of children using the Teaching Strategies 
GOLD (TS GOLD). In addition, most programs surveyed reported supplementing the TS GOLD with 
other measures, including developmental screeners and assessments specific to understanding 
children’s English language proficiency and understanding children’s literacy and language 
development. 

e PEG children, on average, spent nearly forty percent of their time in teacher led activities (including 
academic and creative activities and circle time) and approximately the same percentage of time in 
meals, routines, and transitions. Classrooms varied widely in the proportion of time that children 
spent in these two categories of activities. 

e PEG children, on average, spent the majority of their time in activities where the all children in the 
group were together versus activities where children were in smaller groupings, though the proportion 
of time varied widely from one classroom to the next. 


6.2 Curriculum 


In a recent review of the evidence base, Yoshikawa and colleagues (2013) found support for the efficacy 
of content-specific curricula in increasing children’s kindergarten readiness. Yet, the same review 
acknowledged the limited amount of research on the effectiveness of global curricula. There is generally a 
lack of strong research that speaks to the effectiveness of any one preschool curriculum over another on 
improving children’s school readiness, but the mere presence of a curriculum is widely accepted to be 
necessary for quality preschool environments. 


Curriculum and Learning is one of the four standards considered as part of the Massachusetts Quality 
Rating and Improvement System (QRIS). To meet standards for Level 2 and higher in the QRIS, 
programs must use a developmentally-appropriate, comprehensive curriculum aligned to the state’s 
standards, which all PEG classrooms were expected to do as well. 


All of the PEG classrooms reported using at least one published curriculum as their primary curriculum 
(Exhibit 6.1). Six different curricula were used as the primary curriculum in PEG classrooms. Three of 
these were comprehensive curricula that covered math and literacy: Creative Curriculum, Splash into 
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PreK, and Big Day for Pre-K. Other classrooms used curricula that were literacy-focused (Opening the 
World of Learning (OWL)), and still other classrooms used both OWL and a math-focused curriculum 
(Building Blocks for Mathematics). For more than half of teachers (54 percent), 2015—2016 was their first 
year using their program’s primary curriculum. A few classroom teachers also reported using a variety of 
other secondary curricula. As shown in Exhibit 6.1, these secondary curricula typically included two 
general curricula--Creative Curriculum as a supplementary curriculum and Frog Street PreK, a literacy- 
focused curriculum (Fundations) and a math-focused curriculum (Everyday Math). 


Exhibit 6.1: Primary and Secondary Curricula Used in PEG Classrooms 
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READS AS: Creative Curriculum was the primary curriculum in 12 classrooms and a secondary curriculum in nine other classrooms. 
Notes: Dark blue = primary curriculum. Light blue = secondary curriculum. 

SAMPLE: N= 48 classrooms. 

SOURCE: PEG Teacher Survey (Spring 2016). 


In three of the PEG communities, selection of the primary classroom curriculum was a community-level 
decision; the same primary curricula were used by all classrooms in all ELPs (although classrooms in the 
same community also reported using different secondary curricula). In two communities, different ELPs 
used different curricula. Across the communities, about one-third of the PEG classrooms reported using 
different secondary curricula. 


e In Boston, all PEG classrooms implemented OWL and Building Blocks. 

e In Springfield, all PEG classrooms used Big Day for PreK as well as different secondary curricula. 

e In Lowell, all PEG classrooms used Creative Curriculum. 

e In Lawrence, PEG classrooms from one ELP used Splash into PreK and classrooms from the second 
ELP used Creative Curriculum. 


e In Holyoke, PEG classrooms from one ELP used Creative Curriculum and classrooms from the 
second ELP used OWL. 
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6.3. Screening and Assessment 


Little research has focused on the effects of child assessment in preschool, yet research in other grades 
suggests that ongoing assessment of children’s progress by teachers may improve academic skill 
acquisition and kindergarten readiness (for a review of this literature, see Atkins-Burnett et al., 2014). 
Assessing children’s progress is important for teachers to be able to identify where each individual child 
is at regular intervals during the program year, to provide appropriate instruction, and to monitor their 
progress as their year passes. Some curricula include specific directions regarding the timing and manner 
in which children’s progress should be monitored and/or assessed. In addition to guidance from the 
curriculum, preschool programs often require teachers to use additional assessment tools. Data collected 
by teachers about children’s learning and development can be used to individualize instruction, 
redistribute program or classroom resources, and focus teacher outreach to and engagement of parents. 


All PEG classrooms were expected to use Teaching Strategies GOLD (TS GOLD) to assess children’s 
progress across multiple developmental domains. All PEG classrooms reported conducting screening and 
assessments of children at enrollment and throughout the 2015—16 year with TS GOLD. Just over half of 
the classrooms also reported using additional screening/assessment tools in just over half of the 
classrooms (26 of 48) (Exhibit 6.2). The Ages & Stages Questionnaire (ASQ), a screening tool that 
measures children’s progress toward developmental milestones, was the most common additional 
screening tool used in PEG classrooms (17 classrooms). The DIAL was used as an additional assessment 
tool in 6 classrooms, the PPVT in 3 classrooms, the Brigance and PELI in two classrooms each, and the 
DIBELS and Ages and Stages Socio-emotional Questionnaire in a single classroom each. Two English 
language proficiency screeners were used by some of the PEG programs. The WIDA was used in four 
classrooms in one of the communities and the preLAS in a single classroom in a second community. 
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Exhibit 6.2: Additional Assessment/Screening Tools Used in PEG Classrooms (in addition to TS 
GOLD) (n=35 classrooms) 
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READS AS: Ages and Stages was used for assessment/screening in 17 classrooms. 

NOTE: Grey bars represent general screening tools. Red bars represent English language proficiency screening tools. From left to right, Ages 
and Stages (Ages and Stages Questionnaires); DIAL (Developmental Indicators for the Assessment of Learning); PPVT/EVT (Peabody Picture 
Vocabulary Test/Expressive Vocabulary Test); PELI (Preschool Early Literacy Indicators); Brigance Early Childhood Screen); DIBELS 
(Dynamic Indicators of Basic Early Literacy Skills); Ages and Stages-Socio-emotional (Ages and Stages Questionnaires: Socioemotional); 
WIDA (World Class Instructional Design and Assessment) assessment of English language proficiency), preLAS (English language proficiency 
screener); 

SAMPLE: N=35 classrooms. 

SOURCE: PEG Teacher Survey (Spring 2016). 


PEG teachers reported using the information gathered from these assessments for a number of purposes 
(Exhibit 6.3). Individualizing instruction and communicating with parents about their child’s progress 
were the most common ways that PEG lead teachers reported using screening and assessment data. 
Approximately half of lead teachers reported using assessment data to inform referrals to special 
education or other additional services, and one-quarter of teachers used these data to provide their 
program and district leadership with information on children’s progress. 
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Exhibit 6.3: Uses of Child Screening and Assessment Data in PEG Classrooms 
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READS AS: Eighty-three percent of teachers reported using child screening and assessment data to provide information to parents. 
SAMPLE: N=38. 
SOURCE: PEG Teacher Survey (Spring 2016). 


6.4 Classroom Activities and Groupings 


How and where children spent their time in PEG classrooms was recorded by the study team using the 
OMLIT Classroom Snapshot; the observations lasted for four hours (240 minutes), on average, during the 
morning. Each child’s activities were categorized by type of activity (teacher-led activities, child-directed 
activities, meals routines and transitions, and gross motor activity). While each activity observed was 
assigned to a single category in terms of its goal, many activities likely served multiple goals in the 
observed classrooms. For example, playing with blocks was coded as a creative activity, yet this activity 
may have also supported children’s understanding of mathematical concepts. Within each of these 
activities, the involvement of the teacher and the size of the groupings of children (children on their own 
or in small, medium or large groups) were also recorded. 


6.4.1. Classroom Activities 


Teacher-led activities accounted for the largest amount of children’s time in PEG classrooms during the 
morning. Over the four hour observation period, children in PEG classrooms spent one and a half hours 
(38.5 percent of classroom time), on average, in these teacher-led activities (Exhibit 6.4). This category 
includes time in which teachers and children were engaged in academic activities such as reading, math, 
science, and writing; circle time, which typically includes math and literacy activities as well as science 
and nature activities; and creative activities such as dramatic play, art, music, block play, fine motor play, 
sensory play, and games. Among these activities, reading were the most common teacher-led activity (11 
percent of classroom time) followed by creative play (9 percent of classroom time) and circle time (9 
percent of classroom time). Activities involving mathematics accounted for about 7 percent of children’s 
classroom time. Science and writing related activities were less common, on average (2 percent and 0.7 
percent of classroom time respectively). 
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Exhibit 6.4: Percentage of Time Spent by PEG Children by Type of Activity 
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READS AS: PEG children spent an average of 38.5 percent of their time on teacher-directed activities during the 4 hour observation period. 
SAMPLE: N=48. 
SOURCE: PEG Classroom Observations (Winter 2016). 


There was substantial variation across the PEG classrooms in time spent in teacher-led activities. In three 
classrooms children spent less than an hour (25 percent of classroom time) engaged in teacher-led 
activities, while in six classrooms, these activities accounted for more than 2 hours (50 percent) of 
children’s time. 


Meals, routines, and transitions represented the second most common use of time in PEG classrooms. 
Children in PEG classrooms spent nearly one and a half hours (37 percent of classroom time) engaging in 
meals, routines and transitions, although there was substantial variation by classroom. Time spent ranged 
from as little as 43 minutes (18 percent of time) in routines up to more than two hours and 45 minutes (69 
percent of classroom time). 


About half of the time that children spent in routines and transitions included verbal communication 
between the teacher and children. Some communications were for the purpose of management, for 
example, Everybody in the room needs to go to the cubbies and get their jackets for outdoor play time” 
and some communications were for the purpose of supporting children’s learning. In about one-quarter of 
the classrooms, teachers were observed to use these transition times as an opportunity for literacy-related 
activities. As examples, in one classroom, teachers used index cards to practice phonics or played 
rhyming games with the children as they waited in line, and in another classroom, teachers read books to 
children during transition times. In the classrooms that infused literacy activities into transition time, time 
spent doing literacy activities ranged from 8 to 33 percent of the time spent in routines and transitions. 


The remainder of children’s time in the PEG classrooms was split between child-directed activities 
(which could include creative activities, reading, math or science activities, or other activities). These 
accounted for 45 minutes or 13 percent of time. Children were also involved in gross motor play for on 
average 30 minutes during the morning. PEG children spent little time watching television or using 
computers. 
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6.4.2 Child Groupings 


PEG children spent the majority of their time -- more than two and one half hours (64 percent) of the 
morning--in activities in which all children were doing the same activity (whole group). Children spent 
23 percent of their time in medium groups (defined as four or more children but not all students), and 

13 percent of their time in small groups (defined as three or fewer children). Meals, routines and 
transitions were the most common whole group time activities (48 percent of all whole group time). 
Creative and academic activities most often were conducted with children in small groups. Although all 
PEG classrooms spent a substantial amount of time in whole group activities, there was variation across 
classrooms. Classrooms ranged from 40 percent of children’s time in whole group to nearly 88 percent of 
time (Exhibit 6.5). 


Exhibit 6.5: Percentage of Time Spent by PEG Children in Different Group Sizes 
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READS AS: PEG children spent an average of 13 percent of their time in small groups. 


SAMPLE: N=48. 
SOURCE: PEG Classroom Observations (Winter 2016). 
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7. Classroom Quality 


As discussed in Chapter 1, the PEG program model is designed to provide quality early childhood 
education to low-income four-year-old Massachusetts children. For the PEG program, the meaning of 
quality encompasses multiple components including teacher supports, parent engagement and family 
supports, community collaboration, and classroom instructional quality. This chapter focuses specifically 
on the classroom quality component. Previous chapters of this report have presented information about 
the extent to which PEG programs provided the supports hypothesized to be key ingredients in producing 
classroom instructional quality, including teacher compensation, teacher professional development, 
coaching and paid planning time, and the use of developmentally appropriate curricula and assessments. 


99 Gos 


Current conceptualizations of classroom quality differentiate “process quality,” “instructional quality”, 
and “structural quality.” Process quality includes teacher warmth, and sensitivity, and classroom 
organization and management. Instructional quality includes the type of programming (curriculum with 
scope and sequence versus teacher-selected activities), whether the instructional programming includes 
domain-specific instruction, the extent to which instruction is differentiated, extent of teacher scaffolding, 
provision of a variety of age-appropriate, engaging activities. Structural quality includes teacher-child 
ratio, class size, class composition, time allocation, and classroom culture. The PEG program targeted all 
three aspects of quality, through its teacher PD and coaching, adoption of standardized curricula, and 
requirements on class size and ratio. 


This chapter focuses on both process and instructional quality, using three standardized classroom 
observation instruments to describe PEG classrooms. This chapter presents findings from classroom 
observations with these measures and also includes some relevant findings from recent early childhood 
education studies that used one of the same measures. 


7.1. Summary of Findings 


Key findings from the classroom observations include: 


e The overall quality of the PEG classrooms, on average, was rated as moderate to high. PEG 
classrooms had higher ratings on measures of the quality of the classroom environment compared to 
their ratings on the quality of classroom instruction. 

e Classroom quality varied widely across programs and classrooms. 

e Most PEG classrooms received a mix of moderate and high quality ratings across the three measures. 

e No PEG programs received low quality ratings across all of the quality indicators, and one classroom 
consistently received ratings of high quality across all of the measures. 

e The quality of instruction in PEG classrooms is, on average, comparable to other studies of preschool 
programs serving low-income children that used one of the same measures. 


7.2 Measures of Classroom Quality 


The PEG model includes an implicit assumption that instructional quality will result in greater gains in 
children’s school readiness skills. There is some prior research that suggests that higher-quality child care 
promotes the development of cognitive and academic skills and may promote the development of social 
skills (Pianta, Barnett, Burchinal, & Thornburg, 2009). 
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The quality of the PEG classrooms was directly examined using three standardized observation 
6 
measures: 


e The CLASS-PreK was used to rate overall classroom quality. 

e The ELLCO was used to rate the quality of the literacy environment and literacy instruction. 

e The COEMET was used to rate the quality of the mathematics environment and mathematics 
instruction. 


The ratings of PEG classrooms, on average, represented a moderate to high level of quality (Exhibit 7.1). 
The average CLASS rating for overall classroom quality in PEG classrooms was 4.7 on a 7-point scale. 
The average ELLCO rating was 3.6 on a 5-point scale. The average COEMET ratings for the quality of 
the math environment and instruction were, respectively, 3.9 and 3.5 on a 5-point scale. 


There was substantial variation across PEG classrooms on each of the classroom quality measures, as also 
shown in Exhibit 7.1. Ratings of the PEG classrooms represented nearly the full range of values, from the 
lowest to the highest possible values on each subscale.’ There was also substantial variation in quality 
scores within the five communities as well as within the early learning programs with more than one 
classroom. 


Exhibit 7.1: PEG Classroom Quality Scores, by Measure 
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READS AS: PEG classrooms earned an average total CLASS score of 4.7 out of 7, and classroom total scores ranged from 3.3 to 6.8. 
NOTE: Squares represent average scores for all PEG classrooms. The lines display the range from the highest to lowest score. 
SAMPLE: N=48. 

SOURCE: PEG Classroom Observations (Winter 2016) using the CLASS-Prek, the ELLCO and the COEMET. 


The OMLIT Snapshot was also used to examine how children spent their time in different activities; these data 
are presented in Chapter 6. 


There was also substantial variation in quality scores between classrooms within each of the five communities 
and between classrooms within those early learning programs. See Appendix A-1 for graph showing the class- 
to-class variation of this variation. 
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Exhibit 7.2 displays the percentage of classrooms that received low, moderate, and high scores on the 
CLASS, ELLCO and COEMET subscales.* Most PEG classrooms earned scores that were in the 
moderate to high range. Across each of the observation measures, nearly all PEG classrooms earned 
scores in the moderate and high categories on those subscales that focused on aspects of the Classroom 
Environment. For example, approximately 80 percent of the classrooms earned ratings in the high 
category on the COEMET Math Classroom Culture and ELLCO Classroom Environment subscales. All 
PEG classrooms earned scores in the high or moderate categories on the CLASS Emotional Support and 
Classroom Organization subscales. Fewer classrooms earned high ratings, and some classrooms earned 
low ratings, on the subscales that emphasize aspects of Teacher Behavior and Teacher-Child Interaction. 
On the CLASS Instructional Support subscale, 11 classrooms (23 percent) had ratings in the low 
category. On the ELLCO Language and Literacy subscale, four classrooms (8 percent) had ratings in the 
low category. 


This classifications are based guidance from the measure developers. On the 7-point CLASS subscales, scores 
of 1 or 2 were categorized as Low, scores of 3, 4 or 5 as Moderate and scores of 6 or 7 as High. For the 5- point 
ELLCO, the study team categorized scores of 1 or 2, which are labeled as Deficient and Inadequate, as Low, a 
score of 3 (Basic) as Moderate and scores of 4 (Strong) or 5 (Exemplary) as High. The COEMET does not 
provide similar guidance regarding how to categorize scores. We classify COEMET using the same procedure 
used in classifying ELLCO: scores of 1—2 are labeled Low, score of 3 are labeled Moderate and scores of 4 and 
5 are labeled High. 
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Exhibit 7.2: Number of PEG Classrooms with Low, Moderate and High Quality Ratings by Measure 


CLASS: Classroom Organization Avg = 5.2 out of 7 


CLASS: Emotional Support Avg = 5.7 out of 7 
CLASS: Instructional Support Avg = 3.2 out of 7 
ELLCO: Classroom Environment Avg = 3.9 out of5 
ELLCO: Language & Literacy Avg = 3.4 out of 5 
COEMET Math Activities 


Avg = 3.5 out of 


COEMET Math Classroom Culture Avg = 3.9 out of 5 


0% 20% 40% 60% 80% 100% 
Low Quality Moderate Quality m= High Quality 


READS AS: Researchers observed that 23 percent of PEG classrooms received low quality ratings on instructional support, as measured by 
the CLASS tool. 

SAMPLE: N=48. 

SOURCE: PEG Classroom Observations (Winter 2016) using the CLASS PreK, the ELLCO and the COEMET. 


Classrooms received a range of ratings within and across the three measures. Across the seven subscales 
of the CLASS, ELLCO, and COEMET, most PEG classrooms had a mix of either moderate and high 
ratings (35 classrooms) or low, moderate, and high ratings (12 classrooms); only one classroom had the 
same rating on all of the subscales (which was high). 


7.3 Quality of Classroom Instruction in PEG Compared to Similar Programs 


To help interpret the levels of CLASS scores in the PEG program, the study team compared average 
CLASS scores for PEG classrooms to some other recent studies of early childhood programs serving 4- 
year-olds that overlapped with PEG in terms of being in Massachusetts or in national samples of Head 
Start, community based, and school district programs. Comparing PEG classrooms to two earlier 
Massachusetts studies of representative state samples of public school and community-based preschool 
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settings,” PEG classroom scores were very similar to the scores from the Massachusetts samples on 
Emotional Support and Classroom Organization. PEG scores on Instructional Support were higher 
compared to Instructional Support scores in the Massachusetts UPK study (where higher is defined as 
being at least 0.5 points on the 7-point scale). 


The CLASS scores for PEG classrooms were also compared to scores from two national samples of early 
childhood programs, one a large sample of 671 preK classes operated by different auspices (Burchinal et 
al., 2010) and the other a sample of 277 Head Start programs. The differences between the samples were 
not consistent; PEG classrooms had higher scores on Classroom Organization compared to the national 
sample of preK programs and lower scores on Emotional Support compared to the Head Start sample. 
There were not notable differences in Instructional Support. 


Finally, CLASS scores for PEG classrooms were compared to the scores from two samples of preschool 
classrooms operated by the Boston Public Schools (BPS).'° While the three samples were similar on 
Emotional Support and Classroom Organization, PEG classrooms had lower scores on Instructional 
Support compared to both of the Boston samples. 


One study reported quality on a sample of 115 preschool classrooms that participated in the Massachusetts 
QRIS (Dahlke et al., 2014) and the other study reported on a statewide sample of 93 preschool classrooms in 
which at least 50 percent of students were low-income (Goodson et al., 2010). 


One sample of Boston preschool programs included 10 public school classrooms that were part of the Boston 
K1 in Diverse Settings (K1DS) program (Yudron and Weiland, 2016). The second sample included 83 
preschool programs run by the Boston Public School (Weiland & Yoshikawa, 2013), 
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8. Family Engagement Activities and Comprehensive Services 


Family engagement, or program efforts to connect with families to involve them in their children’s 
education, is an important component of the PEG program model. For many families, the PEG program 
represented the first time that children and parents had participated in an organized early education 
program. PEG programs are also expected to provide or refer families to comprehensive services to 
support children’s physical and mental health and school readiness, as well as overall family social 
service needs (e.g., housing, nutrition, and parent employment). 


The PEG model assumes that efforts to engage families will be beneficial to children’s learning and 
development in two ways: first, family engagement activities will build a connection between the home 
and the preschool program, which will be a positive support for children’s learning and development, and 
second, some of the PEG family engagement activities are designed to support parents in being more 
active teachers of their children in the home. Also, the model assumes that parents who are strongly 
connected to and supportive of their child’s education in preschool will be more likely to continue to 
engage with and advocate for their child’s education in kindergarten and beyond. 


This chapter first describes the family engagement activities in the PEG programs in 2015-16 and next 
describes how PEG programs and teachers have implemented comprehensive service activities, as well as 
how different stakeholders (center directors, teachers, parents) perceived the effectiveness of these 
strategies. The chapter then reports on parent perceptions and behaviors in three areas: parent 
connectedness to the PEG program; parent feelings of self-efficacy in their relationship with their child’s 
teacher and program; and parent involvement in home activities to support children’s learning and 
development. The PEG model assumes that over the course of the preschool year, these perceptions and 
behaviors will improve or increase. In the first year, however, data were only collected from parents once 
during the spring; without a baseline measure at PEG entry, it is not known whether the perceptions at the 
end of the year reflect an increase from where the children would have been without participating in the 
PEG program.'! 


8.1 Summary of Findings 
Family Engagement 


e All PEG programs engaged families during 2015—16 through communications between program staff 
(including but not limited to teachers) and parents focused on the child’s learning and development. 

e Program-parent communication occurred through scheduled parent-teacher conferences, home visits, 
other program events, more informal, ongoing discussions between parents and teachers at drop-off 
and pick-up, and through individual phone calls and in some cases text messages. 

e Most programs (21 of 28 centers) employed a family services coordinator to help promote and 
provide education and engagement opportunities as well as connect families to needed supports. 

e Most programs organized family events such as potlucks, holiday celebrations, and field trips. 
Although parents appeared to enjoy these events, both center directors and parents reported 


Furthermore, the information on parent outcomes was reported via a survey that about half of the PEG parents 
completed, so it is possible that parents who did not participate in the survey are different from those who did. 


Abt Associates Year 1 Massachusetts Preschool Expansion Grant (PEG) Evaluation Report Jj pg. 59 


FAMILY ENGAGEMENT AND COMPREHENSIVE SERVICES 


challenges in scheduling events at times that worked well for families with different work hours and 
family responsibilities. 

e In most programs, parents volunteered in the classroom, although the frequency of volunteering 
varied across programs, from as often as weekly to once per year. 

e PEG center directors reported differing levels of engagement with all parents. About half the center 
directors reported that over 75 percent of parents participated in family engagement activities; other 
center directors reported lower levels of engagement, including some programs that reported that 
fewer than 20 percent of parents were engaged with the program. 


Comperehensive Services 


e Programs varied substantially in how systematically they collected information on family service 
needs. Less than half of centers (12 of 28 centers) used formal needs assessment surveys, whereas 
most centers (24) reported using more informal strategies, such as talking to parents at enrollment or 
during the year. 

e Nearly all PEG centers (26) reported referring families to services, and about half of the programs 
provided some services directly to families. 

e Almost two-thirds of center directors (19 of 28 centers) reported home visits taking place during the 
year. Teachers, family services coordinators, and/or center directors conducted the home visits. 

e Over half of parents (56 percent) reported accessing services such as family nutrition, meal guidance, 
and parenting classes through the PEG program. 

e Multiple program stakeholders reported difficulties coordinating with LEAs to provide services for 
children with special needs. 


Kindergarten Transition 


e Most PEG centers were involved in kindergarten transition planning and provided families with 
assistance in the kindergarten registration process. 


Parent Experiences 


e At the end of the school year, nearly all PEG parents who responded to the survey described 
themselves as connected to their child’s program and confident in their ability to communicate with 
teachers. 

e At the end of the school year, nearly all PEG parents reported having frequent conversations at home 
with their children, and engaging regularly (usually weekly) in activities that supported their 
children’s learning. 


8.2 Family Engagement Activities 


Family engagement efforts are hypothesized to create a greater sense of family members’ connection to 
and trust in the program and increased understanding of child development and ways to support the 
child’s learning and behavior at home. Recent research has indicated that the engagement of families in 
children’s early learning is linked to school readiness and later academic outcomes (Halgunseth, 2009; 
McWayne, Hampton, Fantuzzo, Cohen, & Sekino, 2004; Fantuzzo, McWayne, Perry, & Childs, 2004). 


In 2015-16, PEG programs focused on providing the following types of activities and services for 
families: (a) culturally and linguistically responsive family engagement activities; (b) comprehensive 
services that promoted families’ access to services that support children’s learning and development 
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(vision, hearing, dental, hearing, development and health screenings and referrals and assistance obtaining 
services when appropriate); and (c) support for kindergarten transition. The overarching goal of this work 
was to develop trust that supports program’s ability to support children and their families and to build 
home support for child learning. 


One set of family engagement activities involved communication between the teachers and the parents, 
focused primarily on a child’s developmental progress. A substantial majority of PEG lead teachers 

(87 percent) reported holding regularly scheduled parent-teacher conferences (versus as needed). In these 
centers, most teachers reported that conferences were most frequently several times per year (69 percent) 
or once per year (15 percent). Nearly all teachers reported that they communicated regularly with parents 
about their children at drop-off and pick-up times. In focus groups, teachers in two communities also 
reported using texting to communicate with parents remotely, and in at least one community, teachers 
also sent photos of children engaged in different activities via e-mail or text. 


In centers providing home visiting services (19 of 28 centers), visits were scheduled as often as a few 
times per year (12 of 28 centers), or some reported conducting visits on an annual or as-needed basis (7). 
Approximately one third of PEG centers (9) did not conduct any home visits. Center directors reported 
that PEG teachers (in 14 of 28 centers) and/or family service coordinators (in 13 of 28 centers) most 
frequently were responsible for conducting home visits. Six center directors reported that they were also 
directly involved in leading home visits. 


In circumstances where families may feel uncomfortable about home visits, PEG staff may meet families 
in another nearby location (e.g., community center). One interviewed program director noted that this 
arrangement allows staff to ask questions and engage with parents without making parents feel judged 
about their home (e.g., cleanliness). 


The majority of PEG centers (21 out of 28 centers) who helped to promote and provide education and 
engagement opportunities as well as connect families to needed supports. Teachers and center leadership 
were also often involved in planning family engagement activities. Center- and/or classroom-wide 
activities included family potlucks, field trips, holiday parties, outdoor activities and field trips, and 
multicultural nights. According to center directors, the majority of programs (17) held monthly parent 
events; in the other programs, parent activities were less frequent. Also, almost all centers (26) sent home 
a parent newsletter on a regular basis. About 70 percent of programs (19) had interpreters to help 
communicate with parents who spoke another language, either available at the site on a daily or weekly 
basis (7) or as needed (12). 


About two-thirds of surveyed lead teachers (68 percent) reported that parents also volunteered in the 
classroom. About half of lead teachers (46 percent) reported that parents volunteered a few times during 
the year. Fewer teachers reported that parents volunteered regularly—a few times per month (reported by 
8 percent of lead teachers) or monthly (13 percent). 


Teachers across PEG communities emphasized during focus groups the importance of developing 
relationships with families, and many reported that their outreach efforts helped parents feel more 
comfortable in the program and within the larger education system. Teachers generally were comfortable 
communicating with parents/guardians and maintaining home-school connections; over 80 percent of 
surveyed lead teachers felt very confident in their ability to speak with parents about student progress and 
behavior, communicate through paper classroom newsletters, and describe classroom activities to parents. 
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Across these various outreach activities, PEG center directors reported differing levels of engagement 
among parents, with some programs reporting that only a small number of parents were engaged in the 
program and other programs reporting that all families participated in the program. Overall, 15 of 

28 center directors interviewed (54 percent) reported that more than three-quarters of PEG families were 
engaged in the program. Only two centers reported that 20 percent or less of their parents were involved 
with the PEG program. There was variation in the level of family engagement both across and within 
communities. 


8.2.1. Challenges in Engaging Families 


Programs reported effectively communicating with parents about events and activities, but scheduling 
appeared to be a major barrier to parent attendance. Parents credited teachers for their efforts to inform 
parents through direct face-to-face communication, flyers sent home with children, and electronic 
reminders. However, over half of parents participating in focus groups expressed frustration that school 
events conflicted with their work or school schedules or with other responsibilities at home (e.g., meal 
preparations, etc.). There was no one time that parents agreed would work best for all families to 
participate in program-sponsored activities. Similarly, center directors expressed difficulties 
accommodating parents’ work schedules in their planning efforts, as well as other program-level logistical 
barriers, such as cost, space constraints, planning time, transportation, and providing child care. A few 
center directors noted that some families had little interest in being engaged and were particularly difficult 
to reach despite multiple outreach efforts. 


8.3. Comprehensive Services for PEG Families 


Comprehensive services provided to families have been shown to improve children’s behavior, improve 
attendance, and reduce later need for services in elementary school. Facilitating families’ referral and 
connection to services has become a popular feature of many programs (Gilliam & Zigler, 2001; Irish, 
Schumacher, & Lombardi, 2004). Some programs that have had significant long-term effects on 
children’s health and educational outcomes provide comprehensive education, family, and health services 
in addition to half- or full-day preschool programming (Reynolds, Temple, Robertson, & Mann, 2001). 
Head Start programs similarly have a long history of providing comprehensive services like 
comprehensive health screenings and home visits, which are required under Head Start Program 
Performance Standards. 


8.3.1 Assessing Family Needs 


PEG programs used a combination of strategies to assess family needs. While some centers appeared to 
have a clear system in place to identify family needs and refer families to services, many centers provided 
less guidance and support. About half of PEG centers (12 of 28) systematically collected relevant 
information on family needs using parent surveys/needs assessments. Almost all centers (24) collected 
information through more informal discussion with parents, at the time of enrollment and/or during the 
school year. 


8.3.2 Linking Families to Needed Services 


All PEG center directors reported referring families to services in the community. Centers referred 
families to a variety of services, mostly targeting children’s needs, such as medical, dental and mental 
health services. Centers less frequently referred families to adult-focused services such as GED 
preparation/adult literacy programs, transportation, or immigration services (Exhibit 8.1). 
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Exhibit 8.1: Types of Services to which PEG Families Were Referred 


Service Type Number of Centers (%) 

Mental health/behavioral supports 11 (39%) 
Medical/dental services T (25%) 
Speech therapy T (25%) 
Occupational therapy 3 (11%) 
Physical therapy 2 (7%) 
Services for Families 

Assistance in finding medical care; mental health screenings; physical health screenings 8 (29%) 
Parenting classes T (25%) 
Social services 6 (21%) 
Family nutrition support 6 (21%) 
GED preparation/adult literacy programs 3 (11) 
Psychotherapy 2 (7%) 
Transportation 1 (4%) 
Immigration services 1 (4%) 


NOTE: The services for children may include services for children with IEPs. 

READS AS: Twelve PEG Center directors (or 43 percent of centers) reported referring PEG families to services for children with IEPs. 
SAMPLE: N=28. 

SOURCE: PEG Center Director Interviews (Spring 2016). 


The proportion of families within programs referred for comprehensive services varied by program. 
About half of PEG programs (12 of 28) programs reported that few families (10 percent or less) had 
received comprehensive services either directly from the PEG program or through referrals, and in five 
programs, the majority of parents (more than 75 percent) received services or referrals through the 
program. 


Overall, about two-thirds of the parents who replied to the survey (66 percent) agreed that the PEG 
program had connected them to services that they needed during the year. As noted earlier, about half of 
parents responded to the survey that these data are from, and these parents may or may not be 
representative of the entire population. Surveyed parents reported varying levels of agreement across 
communities regarding the sufficiency of comprehensive service referrals (Exhibit 8.2). Furthermore, a 
significant proportion of parents (44 percent) reported that they had not received any comprehensive 
services through the PEG program. 
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Exhibit 8.2: Parent Agreement that Program Helped Connect to Services Needed, Overall PEG and 
by Community 
100% 
90% 
80% 
70% 
60% 
50% 
40% 
30% 
20% 
10% 


0% 


Overall PEG A B C D E 


Community 
mStrongly Agree m Agree Slightly Disagree Slightly Strongly Disagree =m Very Disconnected 


READS AS: Fifty-four percent of PEG parents reported that they strongly agreed that their child’s preschool program helped connect them with 
services they needed this year. 

NOTE: The wording of the question was: “Overall, how much do you agree that your child’s preschool program has helped connect you with 
the services that your family needed this year? (Select one only))” 

SAMPLE: N ranges from 35 to 103, by community. For overall PEG, N=336. 

SOURCE: PEG Parent Survey (Spring 2016). 


8.4 Services for Children with Special Needs 


Less than one-tenth of parents surveyed (8 percent) reported that their child was referred for special 
education services during 2015-16. Of the parents who received IEP-related services, 70 percent were 
very or somewhat satisfied, 13 percent were neutral, and 17 percent were somewhat or very dissatisfied 
with the services that they received. 


Center directors reported difficulties coordinating IEP-related services with the LEAs. They identified the 
shortage of LEA special education staff and the high demand for services as barriers. Teachers in four 
communities discussed the importance of IEPs in providing children with appropriate resources and 
support, and also expressed disappointment and frustration with the length of the IEP process. During 
focus groups, multiple teachers described children who waited for months without services until the 
correct paperwork was completed. The teachers also noted that public school teachers had more resources 
than they did for children with special education needs. 


Coordination with parents during the IEP process was another challenge that multiple center directors 
noted during interviews. They mentioned that some parents were distrustful of the IEP process or did not 
agree that their child might have a developmental delay or disability. 
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8.5 Kindergarten Transition Supports 


In the first year of PEG program implementation, all of the centers began planning for supports for 
kindergarten transition, and almost all centers had started transition planning (10 of 28 centers) or had 
begun to provide transition services (15) for the first cohort of PEG children. Center directors reported 
that family engagement staff played a large role in coordinating transition-focused events and activities 
like “Kindergarten Night” or “Big Day.” Similar events were also offered by the public schools and 
advertised to PEG families. ELP leadership reported communicating with LEA staff to ensure that all 
families successfully registered, and center directors also reported providing general registration and 
transition support to families. As of the spring before kindergarten, most parents who participated in focus 
groups had already completed the kindergarten registration process, owing in large part, according to 
parents, to the efforts of PEG staff. 


8.6 Parent Experiences 
8.6.1 Parent Connectedness to their Child’s PEG Program 


Efforts to engage parents appear to have been largely successful in making parents feel comfortable with 
and connected to the PEG program in 2015-16. Overwhelmingly, surveyed parents expressed broad 
satisfaction with the program. For example, nearly all parents (94 percent) indicated that they felt 
welcome at their children’s schools. Families also expressed satisfaction with the quality and intensity of 
their communication with teachers; nearly all parents indicated that teachers kept them informed them 
about their child’s progress in school (91 percent) and that teachers were interested in their children and 
cooperative with their families (95 percent). While nearly all parents (93 percent) also felt informed about 
meetings and special school events, somewhat less (81 percent) agreed that parent activities were 
scheduled at convenient times. 


Overall, the vast majority of PEG parents (89 percent) reported that they felt “very connected” or 
“somewhat connected” to the program (see Exhibit 8.3). The level of connection varied by center, ranging 
from centers in which 90 percent of parents reported being very connected to centers in which 66 percent 
of parents reported being very connected. Of note, no parents reported feeling “strongly disconnected” 
from their PEG program. 
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Exhibit 8.3: Parent Connectedness to PEG Program, Overall PEG and by Community 


100% 


80% 


60% 


40% 


20% 


5% 5% 
0% 
0% 
Overall PEG A B C D E 
Community 
mVery Connected = Somewhat Connected Neutral Somewhat Disconnected =m Very Disconnected 


READS AS: Seventy-five percent of PEG parents reported feeling very connected to the program. 
SAMPLE: N ranges from 35 to 105. For overall PEG, N=334. 
SOURCE: PEG Parent Survey (Spring 2016). 


In focus groups, parents identified some specific aspects of the PEG program they were most satisfied 
with. The most common themes that emerged were 1) individualized attention and frequent 
communication with teachers, 2) safe and caring school environments, 3) kindergarten preparation, and 
4) healthy and high-quality meals. Parents across all five communities most frequently spoke about their 
satisfaction with the PEG teaching staff. For example, multiple parents spoke about how much their 
children enjoyed attending PEG programs because of their strong relationships with teachers. One parent 
noted: 


“Teachers really love [students] like they are their own. My kid is ready to go back to school at the 
end of the day when we get home because she wants to see her friends and teachers.” 


Another parent reported that: 


“Fach teacher is very attentive to the child, and they notice anything happening or differences in the 
child’s day and they communicate this to the parents.” 


8.6.2 Parent Self-Efficacy 


One of the central goals of PEG is to promote family self-efficacy in terms of supporting their child’s 
education. As shown in Exhibit 8.4, almost all PEG parents reported feeling positively about their ability 
to communicate with their child’s teacher and in their level of knowledge about what was happening in 
their child’s classroom. Over 90 percent of parents reported feeling confident about their ability to 
communicate with their child’s teacher, their ability to communicate effectively with their child about the 
school day, their knowledge of what their child was learning, and their ability to help their children 
continue their learning at home. A slightly lower proportion of parents reported that they had the skills to 
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help out at their child’s program (83 percent) and that they knew about volunteer opportunities (80 
percent). These perceptions were generally consistent across PEG communities. 


Exhibit 8.4: PEG Parent Self-Efficacy and Knowledge of Program Activities 


Agree Agree Disagree Disagree 
Strongly Agree Somewhat Somewhat Disagree Strongly 

N % % % % % % 
| know how to communicate 335 58 39 2 1 0 1 
effectively with my child’s teacher 
| know how to communicate 336 56 41 2 0 1 0 
effectively with my child about the 
school day 
| know enough about the things my 335 56 36 6 1 0 0 
child is learning to be able to help 
him/her 
| know how to explain things tomy 336 54 44 5 1 1 0 
child if they have questions about 
what they are learning in school 
| know effective ways to contact 334 54 40 4 1 1 1 
my child’s teacher 
| know about special events inmy 336 51 44 5 3 0 0 
child’s program 
| have the skills to help outatmy 334 39 44 13 3 2 0 
child’s program 
| know about volunteering 331 31 49 10 5 5 2 


opportunities at my child’s program 


READS AS: Fifty-eight percent of PEG parents reported strongly agreeing with the statement “I know how to communicate effectively with my 
child’s teacher.” 

SAMPLE: N=331-336. 

SOURCE: PEG Parent Survey (Spring 2016). 


8.6.3. Home Support for Children’s Learning 


PEG parents also reported high levels of engagement with their children in home activities that were 
supportive of children’s development and learning. Parents were asked about their involvement with their 
child in a variety of different types of activities (Exhibit 8.5). The most frequent activity parents reported 
engaging in was talking with children about school, which 93 percent of parents reported doing every day. 
About one-third of parents (35 percent) reported reading to their child every day.'” The majority of 
parents reported that they engaged with their child at least a few times per week for all of the other 
activities listed. The frequencies of home activities reported by parents were similar across the five PEG 
communities. 


In 2007, 40 percent of poor 3- to 5-year-olds were read to every day, compared with 50 percent of children in 
families at 100-199% of poverty, and 64 percent of children in families at 200% of poverty and above. Source: 
Federal Interagency Forum on Child and Family Statistics. America’s Children: Key national indicators of well- 
being, 2009. Federal Interagency Forum on Child and Family Statistics, Washington, DC: U.S. Government 
Printing Office. Based on National Household Education Survey analysis. 
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Exhibit 8.5: Frequency of Family Activities Reported by PEG Parents 


AFewTimes Oncea 
Every Day This Week Week Never 


(%) (%) (%) (%) 
Talk about what happened in school 93 6 1 0 
Teach child letters, words, or numbers 58 36 6 1 
Play with toys or games indoors together 58 34 7 2 
Involve child in household chores like cooking, cleaning, setting the 54 33 9 5 
table, or caring for pets 
Count different things with child 51 38 10 1 
Talk about TV programs or videos 45 38 11 6 
Teach child songs or music 44 40 11 5 
Play counting games like singing songs with numbers or reading 40 46 ail 3 
books with numbers together 
Take child on errands like going to the post office, the bank, or the 39 42 15 3 
store 
Read to child 35 52 11 3 
Play a game, sport, or exercise together 33 43 20 3 
Tell child a story 33 47 17 3 
Work on arts and crafts together 17 47 25 11 
Play blocks together 15 32 29 25 
Play a board game or a card game together 6 41 34 18 


READS AS: Ninety-three percent of PEG parents reported talking about what happened in school with their child every day. 
NOTES: Percentages in each row may not add to 100 because of rounding. 

SAMPLE: N=331-336. 

SOURCE: PEG Parent Survey (Spring 2016). 
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9. Child Assessments 


A primary goal of PEG is to better prepare children for kindergarten and later school success. This 
chapter summarizes findings about 2015—16 PEG children’s performance toward the end of their 
preschool year on direct assessments of early academic skills and teacher ratings of their social-emotional 
development. 


9.1 Summary 


e Inthe summer of 2016, PEG children scored close to what would be expected for children entering 
kindergarten in the following areas: understanding early math concepts, early literacy, and vocabulary 
comprehension. Scores were higher for early math and literacy than for vocabulary. 

e The proportion of children at age level varied widely across PEG classrooms. 

e Though the average student’s vocabulary score is not far below what might be expected, there are 
some students who did not demonstrate the vocabulary skills needed for kindergarten success. 
Vocabulary scores were generally lower for children from homes where English was not the primary 
language spoken. 

e On average, PEG children were rated by their teachers as having age-appropriate social skills in 
managing behavior and emotions and positive relationships. They were rated somewhat lower in 
participating in group situations. Further, though the average scores were in the moderate range, a 
number of children were rated below age expectations for many of the individual social skills, 
primarily in forming relationships with adults and solving social problems. 


9.2 Early Academic Skills 


Toward the end of the first year of PEG implementation (June-July 2016), the research team measured 
academic performance for 331 children across 47 PEG classrooms." Data collectors trained by the study 
team conducted individual assessments using standardized measures of early math, early literacy, and 
vocabulary. Each of the standardized tests of academic skills is normed such that a standard score of 100 
represents the performance of an average student at this age (based on a national sample of children from 
all socioeconomic groups). As previously described in Chapter 3, the standardized assessment measures 
included: 


e Early Math: Woodcock-Johnson III Tests of Cognitive Abilities: Applied Problems Subtest; 

e Early Literacy: Woodcock-Johnson III Tests of Cognitive Abilities: Letter-Word Identification 
Subtest; and 

e Vocabulary Comprehension: The Expressive One-Word Picture Vocabulary Test. 


Prior to beginning the assessments, children who were identified as coming from homes in which English 
was not the primary language were screened on their understanding of English.'* The preLAS was used 


Children in one other PEG classroom were not assessed because the program was unable to collect any parent 
permission forms for the child assessments. 


For most children, the information on home language came from parents at the time of enrollment. For children 
missing this information, teachers provided it based on their knowledge of the child and the child’s home 
environment. 


Abt Associates Year 1 Massachusetts Preschool Expansion Grant (PEG) Evaluation Report Jj pg. 69 


CHILD ASSESSMENTS 


as a screening measure to confirm whether it was appropriate to assess the child in English or if he/she 

should be tested in Spanish. All but five of the 331 children in the sample passed the English language 

screener and were tested in English. The remaining five children were assessed in Spanish with the two 
Woodcock Johnson subtests and with a bilingual version of the vocabulary assessment. 


Average standard scores for the three early academic content areas are shown in Exhibit 9.1 below. On all 
three measures, the average scores were close to the expected standard score for their age (score of 100). 
PEG children scored lower, on average, on the vocabulary measure, than on the other two constructs. 


Exhibit 9.1: Overall PEG Early Academic Skill Scores 


Standard 
Construct Mean Deviation Range 
Early Math 98.13 12.56 59-133 
Early Literacy 99.40 13.46 60-155 
Vocabulary 90.64 15.71 50-131 


READS AS: PEG children scored a mean of 98.13 on the early math construct, with a standard deviation of 12.56. Scores ranged from 59- 
133. 

SAMPLE: N=324-326. 

SOURCE: PEG Child Assessment Data(Summer 2016) using the Woodcock-Johnson Ill Tests of Cognitive Abilities: Applied Problems Subtest 
for early math skills, the Woodcock-Johnson Ill Tests of Cognitive Abilities: Letter-Word Identification Subtest for early literacy skills and the 
Expressive One-Word Picture Vocabulary Test for vocabulary skills. 


Most PEG children, on average, appeared to have the early math and early word/letter recognition skills 
that will be expected of them once they began kindergarten compared to the national distribution (Exhibit 
9.2). However, over one-third of PEG children scored low or very low on the vocabulary measure, which 
is more than expected in a sample of children this age. Children who enter kindergarten behind where 
they are expected to score given their age can have a difficult time making progress without significant 
intervention in elementary school. 
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Exhibit 9.2: Overall Distribution of PEG Children on Standardized Assessments 


: bas Avg. score 
National Distribution 2 =100 
Early Math (PEG distribution) 2 a 
Early Literacy (PEG - Avg. score 
distribution) |" 10% =99.4 
Vocabulary (PEG Avg. score 
distribution) =90.6 
0% 20% 40% 60% 80% 100% 


Very Low Low m Expected Range mHigh 


READS AS: Two percent of PEG children scored very low on the early math construct, which is consistent with the percentage of children 
receiving a very low score nationally. Nine percent of PEG children scored low on the early math construct, which is fewer than the 14 percent 
of children in the national distribution that scored low on the early math assessment. 

NOTE: The national distribution represents the distribution of scores on standardized measures with mean = 100 and standard deviation = 15. 
‘Very Low”: More than 2 standard deviations below the expected mean of 100 (<70). 

“Low”: Within 2 standard deviations below the expected mean of 100 (70-84). 

“Expected Range”: Within 1 standard deviation above or below the expected mean of 100 (85-115). 

“High”: More than 1 standard deviation above the expected mean of 100 (>115). 

SAMPLE: N=324-326. 

SOURCE: PEG Child Assessment Data(Summer 2016) using the Woodcock-Johnson III Tests of Cognitive Abilities: Applied Problems Subtest 
for early math skills, the Woodcock-Johnson III Tests of Cognitive Abilities: Letter-Word Identification Subtest for early literacy skills and the 
Expressive One-Word Picture Vocabulary Test for vocabulary skills. 


There were substantial classroom-to-classroom differences in the numbers of children who scored low on 
the assessments. In more than half of PEG classrooms (51 percent), none of the assessed children earned 
“low” scores on the test of early mathematics or early literacy, whereas in nearly one-quarter of PEG 
classrooms (17 percent and 23 percent respectively), 25 percent or more assessed children earned low 
scores (see Exhibit 9.3). On the measure of vocabulary, 9 percent of PEG classrooms had no assessed 
children who earned low scores, whereas in 74 percent of PEG classrooms, 25 percent or more assessed 
children earned low scores. There were two classrooms (each from a different ELP) in which none of the 
children earned low scores on any of the measures and four classrooms in which greater than one- quarter 
of assessed children earned low scores on all of the measures. 
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Exhibit 9.3: Percentage of Classrooms with PEG Children Scoring Low by Skill 


Early Literacy 


Classrooms in which 0 
students scored low 


Early Mathematics 17% 
m Classrooms in which 1 %- 
24% of students scored low 


Vocabulary = Classrooms in which 25% or 


greater students scored low 


0% 20% 40% 60% 80% 100% 
Percentage of PEG Classrooms 


READS AS: In 23 percent of PEG classrooms, 25 percent or more of assessed PEG children scored low on the early math construct, 

NOTE: “Low”: More than 1 standard deviations below the expected mean of 100 (<85). 

SAMPLE: N=324-326. 

SOURCE: PEG Child Assessment Data(Summer 2016) using the Woodcock-Johnson Ill Tests of Cognitive Abilities: Applied Problems Subtest 
for early math skills, the Woodcock-Johnson Ill Tests of Cognitive Abilities: Letter-Word Identification Subtest for early literacy skills and the 
Expressive One-Word Picture Vocabulary Test for vocabulary skills. 


9.2.1. Academic Achievement and Home Language 


The study team examined whether the early academic achievement scores were different for the children 
from homes where the primary spoken language was not English. In the sample of PEG children selected 
for the assessments, over one-third of children lived in homes in which English was not the primary 
spoken language. Spanish was the predominant home language (other than English), and there were 
smaller numbers of children from homes where other languages were spoken, including but not limited to 
Chinese, Haitian Creole, Khmer, and Portuguese. Just over 30 percent of the sample of 331 children (104) 
were administered the preLAS screener for English proficiency based on parent identification of home 
language and/or teacher identification. Yet, as already noted, only five children failed the screener of the 
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88 children who it was administered to and therefore received the Spanish or bilingual version of the 


assessments. " 


Exhibit 9.4 shows average standard scores on each early academic achievement measure for the two home 
language groups (English and Spanish). The average difference between groups was most pronounced for 
the vocabulary assessment; children from homes where English was not the primary language scored 
significantly lower than children from homes where English was the primary language. It should be noted 
that the five children who were tested with a bilingual version of the vocabulary measure scored similarly 
to the national norm for children from second language homes; this comparison suggests that the children 
in the sample who were tested in English and were from homes where other languages were spoken could 
have stronger vocabulary skills in their home language compared to English. 


Exhibit 9.4: Early Academic Assessment Scores by Home Language 


145 


130 


Standard Score 
= 


85 
70 
55 
Non English — English (N=128) | Non English = English (N=128)} NonEnglish — English (N=127) 
(N=98) (N=98) (N=97) 
Early Math Early Literacy Vocabulary 


READS AS: On average, PEG children from homes where English was not the primary language scored 95.67 on the early math construct and 
PEG children from homes where English was the primary language scored an average of 99.9. 

SAMPLE: For PEG children from homes where English is not the primary language, N=98, missing=0-1. For PEG children speaking English at 
home, N=127-128. 

SOURCE: PEG Child Assessment Data (Summer 2016) using the Woodcock-Johnson III Applied Problems Subtest/Woodcock Munoz 
Problemas Applicados for early math skills, the Woodcock-Johnson III Letter-Word Identification Subtest/Woodcock Munoz Identificacion de 
letras-Palabras for early literacy skills, and the Expressive One-Word Picture Vocabulary Test (English only) for vocabulary skills. 


Children with lower scores on the preLAS, which indicate weaker English language abilities, also scored 
lower on the EOWPVT. This relationship indicates that even children with sufficient English language 


'S Children with home languages other than English or Spanish who were identified by program staff as not 
appropriate for the English assessments were not screened during 2015—16. Moving forward, all children 
regardless of home language will be screened using the preLAS. 
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skills to pass the screener may still face barriers to developing and/or demonstrating vocabulary 
understanding that would be expected for children entering kindergarten. 


It should be noted that the assessment scores are from the end of the 2015-16 preschool year and do not 
take into account where children started out at the beginning of the year. Children from non-English 
speaking homes may have started the year lower than their peers from English-speaking homes, although 
this is not known. It also cannot be known from these data whether those children made greater gains 
throughout the year despite scoring below their peers on two of the measures in the summer.'° 


9.3. Teacher Reports of Children’s Social-Emotional Skills 


In addition to early academic outcomes, there is increasing importance placed on non-cognitive skills by 
researchers, practitioners, and policy makers. PEG teachers rated children’s socio-emotional skills using 
the Teaching Strategies GOLD (TS GOLD) at least once during the 2015-16 school year. The Social- 
Emotional section of the TS GOLD that was used in the PEG programs includes nine skills grouped into 
three primary Objectives: 1) self-regulation of emotions and behaviors; 2) positive relationships; and 3) 
participation in group situations. For each of the nine skills, a student is rated on a scale from | to 9. 
Individual skill scores are then averaged into the appropriate Objectives. 


The study team examined teacher ratings from spring of the 2015—16 preschool year for 606 students 
from 40 PEG classrooms"” (see Exhibit 9.5). Average scores for all three Objectives were moderate. 
Scores were lowest for Participation in Group Situations and highest for Positive Relationships. A full 
summary of average scores across the nine skills can be found in Appendix B. 


Exhibit 9.5: Teaching Strategies GOLD Ratings for Overall PEG 


Standard 
Objective Mean Deviation Range 
1. Self-regulation of emotions and behaviors 6.26 1.12 1.3-9.0 
2. Positive relationships 6.44 1.15 2.0-9.0 
3. Participation in group situations 5.77 1.25 1.0-9.0 


READS AS: PEG teachers gave children a mean score of 6.26 for self-regulation of emotions and behaviors, with a standard deviation of 1.12 
and a range of 1.3-9.0. 

SAMPLE: N=606. 

SOURCE: PEG Child Socio-emotional Skills Assessment (Summer 2016) using TS Gold. 


Average scores were not very different for each of the PEG communities. Though there was slight 
variation in average ratings by community, the bulk of variation was between classrooms and, more so, 
between individual students (examined through statistical analysis of the variation in ratings). 


Ratings of individual skills for PEG children can be compared to the minimum ratings that are expected 
for this age range, which vary by skill from 5 to 7. Despite the fact that the average TS GOLD ratings for 


In the evaluation of the Tennessee public pre-k program, researchers reported that English language learners 
started behind their peers at the beginning of pre-k but made greater gains during the year on a composite of 
measures including early math, early literacy, and vocabulary. The English language learners still finished the 
year scoring lower than English-speaking children (Lipsey, Farran, & Hofer, 2015). 


'7 Data provided by EEC included 40 of the 48 PEG classrooms. 


Abt Associates Year 1 Massachusetts Preschool Expansion Grant (PEG) Evaluation Report Jj pg. 74 


CHILD ASSESSMENTS 


all PEG children were moderate, there were sizeable numbers of children who scored below what would 
be expected for a typical student of this age. As Exhibit 9.6 shows, nearly 30 percent of PEG children 
were lower than expected at participating in group situations and 20 percent scored below expectation on 
their skills in regulating their own emotions and behaviors. 


Exhibit 9.6: Distribution of PEG Children on TS GOLD Ratings 


Individual Skill 


Manages feelings 

Follows limits and expectations 

Takes care of own needs appropriately 
Forms relationships with adults 
Responds to emotional cues 

Interacts with peers 

Makes friends 

Balances needs and rights of self/others 


Solves social problems 


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 


Percentage of PEG Children 
Below Expected Range mExpectedRange = mAbove Expected Range 


READS AS: Thirteen percent of PEG children fall below the expected range for how they manage feelings. 
SAMPLE: N=606. 
SOURCE: PEG Child Socio-emotional Skills Assessment (Summer 2016) using TS Gold. 
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10. Discussion of Findings 


The Massachusetts PEG program aims to expand the supply of high-quality preschool opportunities for 
low-income children through collaborations between public school districts and EEC-licensed programs. 
PEG can be seen as a demonstration program and its results — the level of quality PEG programs are able 
to achieve and the impacts on children’s learning — can inform the state about what can be accomplished 
given this level of investment. 


The independent Year 1 PEG evaluation generated evidence about the progress of the PEG program in 
meetings its ambitious set of goals. In general, the PEG program achieved substantial success in a short 
amount of time. By funding existing community-based programs with experience administering preschool 
services, PEG classrooms were generally able to ramp up quickly to enroll children, hire teachers, provide 
full-day, full-year programming, and deliver many of the planned teacher and classroom supports. The 
local collaborative governance structure, which included public school and community-based 
stakeholders, facilitated efforts to align programming across the community. 


The evaluation findings can be used to assess the progress of PEG statewide in meeting its 
implementation goals for increased access, systems change, and delivering high-quality supports for 
educators, families and children. The findings also include a first look at the level of PEG classroom 
quality and the skills of the first cohort of PEG children immediately before kindergarten entry. 


10.1 PEG Classroom Quality 


In the first year of PEG, classroom quality, on average, was moderate to high according to three 
observation instruments. In general, classrooms tended to receive higher ratings for aspects of quality 
related to classroom environment and organization and lower ratings for aspects of quality related to 
teacher instruction. Only a few classrooms were consistently low across their quality ratings; likewise, 
only one classroom received consistently high ratings. 


There is increasing evidence that suggests very high classroom quality is necessary to meaningfully 
improve child outcomes. A recent meta-analysis of studies linking scores on the CLASS with child 
outcomes suggests that scores, particularly in the Instructional Support domain, were more strongly 
related to larger gains in children’s language and literacy skills in higher versus lower quality classrooms 
(Burchinal et al., 2016). Increases in classroom quality predicted gains in language and literacy outcomes, 
but only in high-quality classrooms, across several measures of quality. Similar “threshold” findings were 
not as clear for math and social emotional outcomes. 


Although the level of classroom quality observed in PEG in the first year is promising, it is important for 
PEG stakeholders to develop additional strategies to help classrooms reach the highest levels of quality. 
The classroom quality findings suggest that PEG programs could benefit from more attention to the 
instructional quality of the classrooms in particular; more detailed suggestions are provided later in this 
chapter. 


10.2 Children’s Skills at End of Preschool 


A central goal of PEG is to ensure that the children in the program are prepared for long-term school 
success. Longitudinal research on school performance suggests that the skills children have when entering 
kindergarten strongly predict their academic success in later grades. Therefore, if preschool can make a 
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difference in children’s kindergarten readiness, the model could have long-lasting benefits through 
school. 


The evaluation assessed the cognitive skills of PEG children at the end of their program year. The 
evaluation was not able to conduct similar assessments of the children’s skills at the time they entered the 
program since the evaluation did not start until December 2015. As a result, the study team can describe 
the level of kindergarten readiness of the first cohort of PEG children, but cannot draw conclusions about 
the extent to which the PEG program improved the outcomes for these children. Findings about PEG 
children’s skills at the end of the first year of the program included the following: 


e Atthe end of their preschool year, PEG children, on average, performed at age level on their early 
math and early literacy skills. Performance at this level is notable in light of the fact that all of the 
children come from very low-income households, about one-third come from households where 
English is not the primary home language, and the majority of children not had prior formal early 
education, increasing the likelihood that many children came to the program with little exposure to 
these early academic concepts. 

e Atthe end of their preschool year, PEG children, on average, performed somewhat below age level 
on their understanding of English vocabulary (as assessed by an expressive language assessment); 
one-third of the children scored below age-expectation. 

e Teacher ratings of social-emotional development suggest PEG children developed many of the key 
skills needed in kindergarten, including managing behavior and emotions and forming positive 
relationships. Children lagged in some skills related to functioning in groups, an important area for 
kindergarten success. 


Longitudinal research provides some context about how to interpret the performance of the PEG children 
at the end of their preschool year. The research suggests that there may be long-term negative 
consequences for children who enter kindergarten scoring below average on their early literacy and 
language skills (Layzer and Price, 2008). The growth patterns of children in this longitudinal analysis 
showed that scores on literacy skills in the fall of kindergarten were highly predictive of scores in fifth 
grade. Also, children with high, moderate, and low literacy scores at the start of kindergarten all grew at 
the same rate, on average. Furthermore, the gaps between levels did not narrow over time. As the skill 
tested became more complex (e.g., comprehension versus letter knowledge), the time that it took for 
children to catch up increased, and some children did not catch up even by the end of fifth grade. For the 
skills needed to interpret and understand what is read, which for most children do not manifest themselves 
until the end of the kindergarten year, the gap between children who entered kindergarten with average- 
high skills and below-average skills actually grew over time. Although many children who entered 
kindergarten with lower performance caught up to others on some skills, the delay may cost them the 
opportunity to develop adequately some other skills important for school success. 


It is positive that at the end of the preschool year, PEG children demonstrated levels of early math skills, 
early literacy skills, and vocabulary comprehension close to what would be expected for children entering 
kindergarten. However, the notable proportion of children who did not perform at age expectation in 
vocabulary development is worthy of attention, since research shows that there are lasting consequences 
for children who enter kindergarten scoring below average on their early literacy and language skills. 
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10.3 PEG Implementation Progress in the First Year 


This section discusses the extent to which PEG communities and programs met some of their key goals 
over the course of the first year of implementation. 


Increased Access and Community Collaboration 


In all five communities, the LEA and ELPs successfully met their goal for serving high-risk families. 
PEG families had very low incomes and about one-third of children came from homes where English was 
not the primary language. In four of the five PEG communities, most of the PEG children had not 
previously participated in formal early education. 


PEG communities partially met their goal of effective local cross-sector collaboration. Each of the PEG 
communities established a collaboration of key stakeholders representing the LEA, Head Start agencies, 
and other community programs. PEG leadership from the five communities reported some successes and 
some challenges with their collaborative efforts. Each community plans to continue to build on its efforts 
so far and tackle some of the more challenging issues faced. 


Teacher Supports 


As detailed in Chapter 5, during 2015-16, the PEG communities were still in the process of developing 
their vision for teacher supports. There was substantial variation across communities in the dosage and 
types of teacher professional supports provided. On average, PEG lead teachers were offered 23 hours of 
professional development training in the first year, across a variety of topics. Across PEG communities, 
teachers reported receiving between 10 and 29 hours. Teachers were provided with coaching by LEA 
staff in four of the five PEG communities. The amount of coaching teachers received varied across 
communities and programs. About 44 percent of PEG lead teachers reported receiving more than 20 hours 
of coaching in the first year, while almost one-third of teachers (32 percent) reported receiving 10 or 
fewer hours of coaching. The variation in intensity of teacher supports could have been the result of 
different program approaches to teacher supports and/or partial implementation of planned supports. Over 
the next two years, there will be more information available about the extent to which communities are 
successfully implementing their planned levels and types of teacher supports. 


As PEG programs continue to develop their plans for professional development, the research on teacher 
training is suggestive about strategies more likely to be impactful for teachers. Overall, the findings from 
the evidence base are inconsistent about the extent to which teacher professional development is 
associated with improved program quality and more sensitive teacher interactions with children. The 
evidence of a positive effect is more common in the context of highly-controlled research studies, as 
opposed to large scale training offered by districts or states (see Burchinal, Howes, & Kontos, 2002; 
Burchinal, Cryer, et al., 2002; Clarke-Stewart et al, 2002; Jackson et al., 2006; Lonigan et al., 2011; 
Justice et al., 2016.). There is some evidence that more intensive and longer duration training is more 
likely to yield benefits compared to briefer trainings (Tout, Zaslow, and Berry, 2006; Loeb, Rouse, & 
Shorris, 2007. 


In terms of the content of professional development, a recent threshold study (Burchinal et al., 2016) 
reports that professional development for teachers should target strategies for improving the instructional 
content in teacher—child interactions. Intentional content-specific instruction includes activities designed 
to teach specific skills such as extended discussions with children and systematic feedback to promote 
children’s oral language, phonemic awareness and print knowledge for literacy and numeracy and 
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geometry skills for mathematics. This research suggests that early childhood programs should not only 
emphasize content instruction, but also should target professional development for teachers on these same 
instructional skills. 


The research on the value-added of coaching in combination with teacher training is also inconsistent. On 
the one hand, there are studies that suggest that in-class mentoring in combination with training has no 
added value (Assel et al., 2007; Lonigan et al., 2011). On the other hand, there is a growing body of 
research showing that teacher training that provides the opportunity for on-site practice, with feedback 
from a supportive source, is key to the effectiveness of traditional professional development such as 
workshops, particularly for teachers in classrooms in the lower range of quality (Zaslow, Tout, Halle, 
Whittaker, & Lavelle, 2010; Powell et al., 2010). In 2011, Isner et al. conducted a literature review to 
examine and summarize more than 20 studies on the effects of professional development with a coaching 
element. About 60 percent of the studies reported positive findings on teacher knowledge and attitudes. 
Most of the studies also included a measure of observed quality and practices with children and, of those, 
87 percent reported positive outcomes. Finally, about half of the studies include child outcomes. Of those, 
just over one-half reported positive language and literacy outcomes for children, one-quarter found 
positive outcomes on behavioral measures, and just one study reported a positive outcome on math 
outcomes. Isner concluded that broadly speaking, the existing literature suggests associations between 
coaching and quality improvements in early childhood education. 


Currently, there is little consensus about the features of coaching that are key to effectiveness. Isner et al. 
(2011) also looked for evidence about features of coaching that were most consistently linked with 
positive outcomes for staff and children, including the characteristics of the coaches, coaching activities, 
coordination between coaching and other professional development, dosage of coaching, supervision and 
document/tracking of teacher progress during coaching. The researchers were unable to determine 
patterns of association between identified coaching features and intended outcomes because of the limited 
reporting of the specific features of coaching within the published articles. Similarly, variations in the 
delivery of features could not be described or linked to variations in outcomes, also due to limited 
reporting about specific features. 


Additional research suggests that more intensive and targeted coaching is more likely to produce effects 
on teachers and children. A number of studies find positive effects of targeted coaching on specific 
instructional practices (in contrast to more general coaching on broad instructional principles) (Burchinal 
et al., 2014; Landry, Anthony, Swank, & Monseque-Bailey, 2009; Landry et al., 2011). Another large 
study of professional development (more than 250 early childhood educators in four states) suggests that 
the intensity of coaching matters (where intensive is defined as twice monthly 2-hour sessions. In 
addition, the same study reported that the effect on overall teaching quality is conditional on educators’ 
baseline teaching quality, such that the professional development only benefits educators who begin with 
poor to average teaching quality. 


There is also positive evidence for a coaching model based on the CLASS-- MyTeachingPartner™-- 
which includes web-facilitated consultation and feedback (Hamre, Justice et al., 2010; Pianta et al, 2008); 
this coaching model is both targeted and intensive. Research on the model found positive effects on the 
quality of teacher-child interactions and positive effects on preschoolers’ language and literacy skills 
when teachers used the model with fidelity. Also, a recommendation from the recent threshold study 
(Burchinal et al., 2016) is that programs should not only focus more on content instruction, but should 
also provide aligned teacher training and coaching on the content instruction. 
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It is important to note that in many of these studies, coaches received intensive support. For example, in 
the Landry study, coaching was implemented by two highly trained and closely monitored facilitators, 
who completed a four-day intensive training workshop, participated in weekly conference calls with an 
overall project manager and weekly conference calls with the entire network of facilitators, and were 
observed on-site several times by the manager. 


Given the research, PEG communities and programs could consider a number of ways to try to ensure that 
professional development and coaching is as effective as possible: 


e Plan teacher professional development trainings that are more intensive and longer in duration, as 
opposed to more frequent, less intensive trainings; 

e Improve the alignment of professional development and coaching with the classroom curriculum 
(especially in the context of classroom curricula focused more explicitly on content instruction); 
and 

e Provide more intensive supports to coaches through training, monitoring and supervision, 
communication with other coaches. 


Curriculum and Assessment 


As described in Chapter 6, PEG programs were allowed to choose age appropriate curricula aligned to the 
Massachusetts Curriculum Frameworks, and all of the PEG programs reported using standard, published 
curricula in 2015-16. Some of the curricula used were general early childhood curricula, covering all 
content areas, while others were more content-specific. 


There is increasing emphasis on early childhood curricula that focus on supporting specific aspects (or 
domains) of school readiness. Experimental evaluations of specific curricula have demonstrated moderate 
to large gains for mathematics curricula (e.g., Clements & Sarama, 2008; Weiland et al., 2013), literacy 
curricula (e.g., Bierman et al., 2008; Farver, Lonigan, & Eppe, 2009; Landry, Anthony, Swank, & 
Monesque-Bailey, 2009; Neuman and Cunningham, 2009; Powell et al., 2010; Weiland et al., 2013), 
language curricula (Neuman and Cunningham, 2009; Wasik and Hindman, 2011), and social-emotional 
skills curricula (e.g., Bierman et al., 2008; Fantuzzo, Gadsden, & McDermott, 2011; Raver et al., 2011). 


The evidence is growing that greater attention should be paid to the content of instruction as a key feature 
of instructional quality. A meta-analysis of studies of early childhood education programs (Burchinal et 
al., 2016) finds that a greater overall dosage of center-based early childhood education is related to larger 
gains in academic skills when teachers spend more time in instruction in related content areas. For 
example, gains in mathematics skills are larger when children experience more time in more intensive 
mathematics instruction. 


Given the research, PEG communities and programs could consider ways to intensify their content 
instruction, depending on their current curriculum offerings: 


e Integrate into their programming more explicit content instruction; 

e Improve the alignment of the professional development and coaching with the content instruction, 
including giving teachers opportunities for active practice and modeling; and/or 

e Improve the alignment of curriculum and formative assessment. 
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Family Engagement 


Massachusetts PEG programs, in line with federal requirements for the Preschool Expansion Grants, 
focused on providing the following types of activities and services for families: (a) culturally and 
linguistically responsive family engagement activities; (b) comprehensive services that promote families’ 
access to services that support children’s learning and development (vision, hearing, dental, hearing, 
development and health screenings and referrals and assistance obtaining services when appropriate); and 
(c) support for kindergarten transition. As discussed in Chapter 8, all of the PEG programs began to 
implement activities and services in all three areas, with a variety of approaches and strategies. Nearly all 
of the programs also reported challenges in working successfully with families, both challenges related to 
program operations and service coordination and challenges related to the level of family participation. 


All PEG programs used multiple methods to communicate with parents, including newsletters, emails, 
and brief check-ins, when parents came to the programs to drop off or pick up their children. In addition, 
almost two-thirds of PEG programs conducted at least one scheduled home visit to families, and some 
programs conducted multiple home visits. All programs organized family engagement activities such as 
potluck dinners or holiday events, which although popular, were irregularly attended. 


Although federal requirements call for a broad set of family supports (as do virtually all state child care 
quality rating systems), there is little research evidence that these types of supports are associated with 
better outcomes for children.'* There have been and continue to be many early childhood programs that 
combine parent education/ involvement components with classroom programs for children, based on the 
hypothesis that children’s learning and development will be enhanced by this dual focus. 


There are at least three early childhood programs that have reported significant impacts at the end of 
preschool and into school: the Perry Preschool Program, the Child-Parent Center Education Program, and 
the AVANCE Parent-Child Education program. Each of these programs combined high-quality preschool 
education starting at age 3 with a substantive parent component that included family engagement and 
comprehensive services. 


In the Perry Preschool Program" teachers conducted weekly home visits designed to involve the mothers 
in implementing the preschool curriculum at home. The Child-Parent Center Education Program” 
combines high-quality preschool starting at age 3 with a comprehensive system of services for children 
and parents that promote parent involvement, has reported significant impacts in preschool and 
throughout the school process to high school graduation. AVANCE” provides early childhood education, 
home visits, weekly 3-hour parenting classes, advocacy and support services. However, the fact that these 
programs “bundle” child-focused and parent-focused components makes it impossible to differentiate the 
contribution of each of the components. It is unclear whether providing services in both the home and 


'S Perhaps reflecting the lack of research, a current framework for high-quality prekindergarten programs and 
systems does not include any features related to family engagement or comprehensive services. 


i Weikart, Bond & McNeil, 1978; Schweinhart et al., 2005. 
*° Reynolds, 2000; Reynolds, Temple et al., 2001; 2002; Ou & Reynolds, 2006; Reynolds, Ou et al., 2004. 


>! Johnson et al., 1996; Walker et al., 1995. 
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preschool program is more effective in supporting the development of young children than targeting one 
or the other context alone. 


The findings from a recent meta-analysis (Grindal et al., 2016) that compared early childhood programs 
with and without parenting education are informative. Overall, the programs with parenting education did 
not have stronger effects on children’s outcomes. However, the authors report some suggestive evidence 
about forms of parent education that are more likely to be associated benefits for children, specifically, 
intensive home visits, defined as one or two home visits a month, and parent education with modeling and 
opportunities for active practice. 


Moving forward, PEG communities and programs could consider: 


e Whether it would be worth investing in more intensive parenting education, either through more 
frequent home visits or training that involves opportunities for parents to actively practice new 
interactions and receive feedback on their practice; and/or 

e Shifting resources for activities that have lower attendance into more intensive activities that 
potentially focus on individuals or smaller groups of parents. 


Comprehensive Services 


PEG programs partially met their goal of providing comprehensive services to families; only some of the 
programs undertook efforts to provide services for families on-site and many programs encountered 
challenges with setting up systems to assess family needs and with referring or providing services to all 
families. During 2015-16, the comprehensive services component of the PEG program model was still in 
an earlier stage of development than the other components in many programs. 


Ideally, how comprehensive services are delivered in PEG could be guided by research on effective 
approaches. However, there is virtually no research that has shown significant effects of case 
management interventions alone in early childhood settings. Most of the research focuses on case 
management in health care (medical and mental health), and although there is an active literature on 
standards for case management, there is little reference to a research basis for these standards (Case 
Management Society of America, 2011). The research on case management has reported mostly no 
effects, although the explanation may be as much about the low quality of the services that families are 
referred as about ineffective case management strategies. The limited literature on successful case 
management interventions describe their effective case management as “intensive,” which is defined as a 
combination of low caseloads for a caseworker and frequent contacts between the caseworker and the 
recipient, and “targeted,” which is defined as focusing on specific types of services as opposed to a broad 
range of services (Scott et. al., 2013). 


Similar to the discussion of family engagement above, there are early childhood programs that have 
reported positive effects that include case management in addition to child-centered programming, 
including the Child-Parent Education Center and AVANCE. Again, because these programs “bundle” 
child-focused and parent-focused components, it is impossible to determine the separate contribution of 
case management. 


In terms of comprehensive services, PEG communities and programs might want to consider either 
adopting more fully particular standards for case management or moving toward the depth of services 
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described for the three example programs above. If so, it is likely that there will need to be more 
resources devoted to this component of the model. 


10.4 Teacher and Parent Experiences 


The PEG model assumes that increasing teacher compensation and providing teachers with more 
intensive professional supports will lead to job satisfaction and, over time, higher rates of teacher 
retention. Although retention could not be assessed at the end of the first year of PEG, PEG teachers were 
asked about their perceptions of their jobs and ability to be effective teachers. Overall, PEG teachers 
reported high levels of job satisfaction and high levels of confidence about being able to reach and teach 
all of the children in their classrooms. 


PEG communities generally met their goal for parent connectedness to the PEG program. On average, 
PEG parents expressed a strong level of connectedness to and satisfaction with their child’s program and 
reported feeling confident about their ability to communicate with teachers about their child. Parents 
reported moderate to high levels of involvement in activities at home that could support their child’s 
learning and development, although the study team is not able to conclude whether the levels of these 
behaviors represented an increase since children enrolled in PEG because data were only collected once 
toward the end of the school year. 


10.5 Next Steps for PEG Evaluation 


In future years (beginning in 2016-17), the evaluation will focus on program impacts on children and 
families, as well as continuing to track implementation of the key program components. Using a rigorous 
regression discontinuity evaluation design, the evaluation will estimate the impact of PEG on children’s 
academic and socio-emotional skills. The study will also look at impacts on the level of home support and 
school connectedness for children’s learning and development. At the end of 2016-17, the evaluation will 
be able to look at impacts for a single cohort of PEG children. Impacts on a second cohort of PEG 
children and families will be estimated at the end of the 2017—18 year. Furthermore, beginning in the 
2016-17 school year, the evaluation will also be able to describe the gains children made during their 
preschool year as well as track the 2016-17 PEG children into kindergarten to follow their progress. 


If additional resources become available, an additional study could include a sample of programs across 
the state that represent different levels of key PEG supports (e.g., teacher compensation, professional 
development, coaching) and then examine through a quasi-experimental study whether children’s gains in 
academic and socio-emotional skills over the year are related to particular quality components. This 
evidence would help policy makers and program developers better understand what the program model 
should look like, in an environment of limited resources, to sustain and ultimately scale-up the PEG 
model. 
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Appendix A: Summary of Classroom Quality Scores 


Exhibit A.1: Classroom Learning Assessment System Scores 


Standard 
Mean Deviation Range 
Total Score 47 0.7 3.3-6.8 
Instructional Support 3.2 1.0 1.2-6.7 
Emotional Support 5.7 0.6 4.3-7.0 
Classroom Organization o2 0.8 3.4-6.7 


READS AS: The average Total CLASS score was 4.7 with a standard deviation of 0.7 points. Scores ranged from a low of 3.3 to a high of 6.8. 
SAMPLE: N=48. 
SOURCE: PEG Classroom Observations (Winter 2016) using the CLASS PreK. 


Exhibit A.2: Early Language and Literacy Classroom Observation Pre-K (ELLCO) Scores 


Standard 
Mean Deviation Range 
Total ELLCO 3.6 0.6 2.2-5.0 
Language and Literacy Subscale 3.4 0.6 2.8-5.0 
Classroom Environment 3.4 0.7 1.8-5.0 


READS AS: The average Total ELLCO score was 3.6 with a standard deviation of 0.6 points. Scores ranged from a low of 2.2 to a high of 5.0. 
SAMPLE: N=48. 
SOURCE: PEG Classroom Observations (Winter 2016) using the ELLCO. 


Exhibit A.3: Classroom Observation of Early Mathematics (COEMET) Scores 


Standard 
Mean Deviation Range 
Mathematics Classroom Culture 3.9 0.5 2.4-4.7 
Mathematics Activities 3.5 0.4 2.8-4.6 
READS AS: The average Mathematics Culture score was 43.9 with a standard deviation of 0.5 points. Scores ranged from a low of 2.4 toa 
high of 4.7. 
SAMPLE: N=48. 


SOURCE: PEG Classroom Observations (Winter 2016) using the COEMET. 
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Exhibit A.4: Distribution of CLASS Subscale Scores by ELP 
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Note: PEG classrooms are represented by blue dots. Classrooms are displayed from highest to lowest score within each ELP. ELPs are 
separated by grey bars and are displayed from highest to lowest mean average score for their classrooms. 


Exhibit A.5: Distribution of ELLCO Subscale Scores by ELP 
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Exhibit A.6: Distribution of COEMET Subscale Scores by ELP 
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Note: PEG classrooms are represented by blue dots. Classrooms are displayed from highest to lowest score within each ELP. ELPs are 
separated by grey bars and are displayed from highest to lowest mean average score for their classrooms. 
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Appendix B: Summary of Teaching Strategies GOLD Objective and 


Skill Ratings 


Exhibit B.1: TS GOLD Objective and Skill Ratings, Overall PEG 


Standard 

Objective N Mean Deviation Range 

1. Self-regulation of emotions and behaviors 606 6.26 1.12 1.3-9 
Manages feelings 606 6.02 1.40 2-9 
Follows limits and expectations 606 6.21 1.17 1-9 
Takes care of own needs appropriately 606 6.56 deal 1-9 
2. Positive relationships 606 6.44 1.15 2-9 
Forms relationships with adults 606 7.06 1.12 2-9 
Responds to emotional cues 583 6.02 1.32 1-9 
Interacts with peers 583 6.30 1.44 2-9 
Makes friends 606 6.35 1.38 2-9 
3. Participation in group situations 606 SLIT 1.25 1-9 
Balances needs and rights of self and others 606 5.93 1.38 1-9 
Solves social problems 583 5.61 1.30 1-9 


READS AS: Teachers rated their students at an average score of 6.26 for self-regulation of behaviors with a standard deviation of 1.12. Scores 
ranged from 1.3 to 9. 

SAMPLE: N=583- 606. 

SOURCE: PEG Child Social-Emotional Skills Assessment (Summer 2016) using TS Gold. 
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